Skip to content

Conversation

@purplesyringa
Copy link

#116 added rayon support, but parallelized only across shards, since std's hashmap does not support parallelization. Since dashmap exclusively uses hashbrown since #259, we can use hashbrown's rayon support for parallel iterators inside the hashmap.

This sounds like an obvious win in theory, but I'm not very familiar with rayon and I'm not sure if there could be some overhead from scheduling. Perhaps @cuviper could say if this is what he had in mind when writing the "implementation note" comment?

@cuviper
Copy link
Contributor

cuviper commented Dec 29, 2025

Yes, that was the intent of my comment.

It's still tough to judge if we should do this though. If the shards are reasonably well balanced to begin with, then the inner parallelism won't help much, and might even hurt a little with the additional overhead of more parallel splits. It would be good to collect some measurements if there are any dashmap/rayon users out there with benchmarks.

@xacrimon
Copy link
Owner

xacrimon commented Dec 31, 2025

Yeah, I'm happy to merge this if there's high quality benchmarks to support it. I don't have the time to perform such a study myself at the moment and my gut feeling is that this is going to be at best a net zero. Rayon scheduling does have measurable overhead and assuming a sane hash function, shards should have nearly equal element counts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants