Skip to content

Use XOR with shift for cache key calculation #413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kelhusseiny wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Use XOR with shift for cache key calculation #413

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions lib/statsd/instrument/compiled_metric.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,12 +151,10 @@ def self.#{method}(__value__#{default_val_assignment}, #{tag_names.map { |name|
__return_value__ = StatsD::Instrument::VOID
#{generate_block_handler if allow_block}

# Compute hash of tag values for cache lookup using rotate-left + XOR.
# Rotation makes it order-dependent (unlike plain XOR), preventing collisions
# when tag values are swapped. We mask to 32 bits to avoid Bignum allocations
# from left shifts on 64-bit hash values.
__cache_key__ = #{tag_names.first}.hash & 0xFFFFFFFF
#{tag_names.drop(1).map { |name| "__cache_key__ = (((__cache_key__ << 5) | (__cache_key__ >> 27)) ^ #{name}.hash) & 0xFFFFFFFF" }.join("\n")}
# Compute hash of tag values for cache lookup using XOR with position-based shifts.
# Shifting each hash by its position makes it order-dependent, preventing collisions
# when tag values are swapped (e.g., (a, b) vs (b, a)).
__cache_key__ = #{tag_names.map.with_index { |name, i| i.zero? ? "#{name}.hash" : "(#{name}.hash >> #{i})" }.join(" ^ ")}

# Look up or create a PrecompiledDatagram
__datagram__ =
Expand Down
8 changes: 4 additions & 4 deletions test/compiled_metric_test.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -285,10 +285,10 @@ def test_emits_metric_on_hash_collision

cache = metric.instance_variable_get(:@tag_combination_cache)

# Compute cache keys using the rotate-left + XOR formula (32-bit bounded)
# For single tag, it's the hash value masked to 32 bits
cache_key_for_1 = 1.hash & 0xFFFFFFFF
cache_key_for_2 = 2.hash & 0xFFFFFFFF
# Compute cache keys using XOR with position-based shifts
# For single tag (position 0), it's just the hash value with no shift
cache_key_for_1 = 1.hash
cache_key_for_2 = 2.hash

# Store the cached datagram under the collision key
cached_datagram = cache[cache_key_for_1]
Expand Down