-
Notifications
You must be signed in to change notification settings - Fork 2
part: mark inner nodes with a transaction id to detect mutability #132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+122
−147
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
63e6fc2 to
f332e6e
Compare
bimmlerd
reviewed
Jan 14, 2026
pippolo84
reviewed
Jan 14, 2026
Member
pippolo84
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a couple of minor nits, overall LGTM.
f332e6e to
7b19b7c
Compare
bimmlerd
approved these changes
Jan 15, 2026
Member
bimmlerd
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor nit, LGTM
pippolo84
approved these changes
Jan 15, 2026
txnID marks the generation of the transaction. This allows checking
if a node had been cloned during this transaction or not which is more
efficient and accurate than with nodeMutated. For iterating queries against
the tree the txnID is bumped so that in-placed mutations are not allowed.
The 'NoCache' option was removed as there was no longer any benefit with it
(it was used earlier with Set[T] where it was known that there was always
just one write per transaction and no point with nodeMutated record keeping).
Deletion now short-circuits if it hits a parent that had already been cloned
and we can stop updating the parent pointers.
Benchmarks:
goos: darwin
goarch: arm64
pkg: github.com/cilium/statedb/part
cpu: Apple M1 Pro
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
_Uint64Map_Random-8 608.4µ ± ∞ ¹ 589.4µ ± ∞ ¹ -3.12% (p=0.008 n=5)
_Uint64Map_Sequential-8 505.0µ ± ∞ ¹ 487.3µ ± ∞ ¹ -3.52% (p=0.008 n=5)
_Uint64Map_Sequential_Insert-8 469.2µ ± ∞ ¹ 448.9µ ± ∞ ¹ -4.33% (p=0.008 n=5)
_Uint64Map_Sequential_Txn_Insert-8 70.41µ ± ∞ ¹ 69.88µ ± ∞ ¹ ~ (p=0.341 n=5)
_Uint64Map_Random_Insert-8 564.1µ ± ∞ ¹ 534.9µ ± ∞ ¹ -5.17% (p=0.008 n=5)
_Uint64Map_Random_Txn_Insert-8 130.5µ ± ∞ ¹ 109.1µ ± ∞ ¹ -16.42% (p=0.008 n=5)
_Insert_RootOnlyWatch-8 73.25µ ± ∞ ¹ 72.76µ ± ∞ ¹ ~ (p=0.310 n=5)
_Insert-8 101.5µ ± ∞ ¹ 104.6µ ± ∞ ¹ +3.05% (p=0.008 n=5)
_Modify-8 59.40µ ± ∞ ¹ 61.82µ ± ∞ ¹ +4.07% (p=0.008 n=5)
_GetInsert-8 76.89µ ± ∞ ¹ 77.75µ ± ∞ ¹ +1.12% (p=0.032 n=5)
_Replace-8 19.89n ± ∞ ¹ 21.21n ± ∞ ¹ +6.64% (p=0.008 n=5)
_Replace_RootOnlyWatch-8 19.90n ± ∞ ¹ 21.20n ± ∞ ¹ +6.53% (p=0.008 n=5)
_txn_1-8 149.4n ± ∞ ¹ 129.0n ± ∞ ¹ -13.65% (p=0.008 n=5)
_txn_10-8 91.14n ± ∞ ¹ 74.24n ± ∞ ¹ -18.54% (p=0.008 n=5)
_txn_100-8 63.68n ± ∞ ¹ 63.86n ± ∞ ¹ ~ (p=1.000 n=5)
_txn_1000-8 73.85n ± ∞ ¹ 72.99n ± ∞ ¹ -1.16% (p=0.048 n=5)
_txn_delete_1-8 474.9n ± ∞ ¹ 180.2n ± ∞ ¹ -62.06% (p=0.008 n=5)
_txn_delete_10-8 96.95n ± ∞ ¹ 69.14n ± ∞ ¹ -28.68% (p=0.008 n=5)
_txn_delete_100-8 63.34n ± ∞ ¹ 60.89n ± ∞ ¹ -3.87% (p=0.008 n=5)
_txn_delete_1000-8 53.08n ± ∞ ¹ 50.18n ± ∞ ¹ -5.46% (p=0.008 n=5)
_Get-8 16.31µ ± ∞ ¹ 16.90µ ± ∞ ¹ +3.63% (p=0.008 n=5)
_All-8 4.654µ ± ∞ ¹ 4.552µ ± ∞ ¹ ~ (p=0.056 n=5)
_Iterator_All-8 4.578µ ± ∞ ¹ 4.562µ ± ∞ ¹ ~ (p=0.841 n=5)
_Iterator_Next-8 4.846µ ± ∞ ¹ 4.935µ ± ∞ ¹ +1.84% (p=0.008 n=5)
_Hashmap_Insert-8 37.74µ ± ∞ ¹ 39.55µ ± ∞ ¹ +4.79% (p=0.008 n=5)
_Hashmap_Get_Uint64-8 5.693µ ± ∞ ¹ 5.734µ ± ∞ ¹ ~ (p=1.000 n=5)
_Hashmap_Get_Bytes-8 6.111µ ± ∞ ¹ 6.327µ ± ∞ ¹ +3.53% (p=0.032 n=5)
_Delete_Random-8 40.14m ± ∞ ¹ 11.72m ± ∞ ¹ -70.80% (p=0.008 n=5)
_nodeMutatedClear-8 7.101n ± ∞ ¹
_nodeMutatedExists-8 3.870n ± ∞ ¹
_find16-8 3.492n ± ∞ ¹ 3.491n ± ∞ ¹ ~ (p=0.762 n=5)
_findIndex16-8 8.831n ± ∞ ¹ 8.808n ± ∞ ¹ ~ (p=0.095 n=5)
_find4-8 2.077n ± ∞ ¹ 2.079n ± ∞ ¹ ~ (p=0.175 n=5)
_findIndex4-8 2.837n ± ∞ ¹ 2.832n ± ∞ ¹ -0.18% (p=0.024 n=5)
geomean 1.678µ 2.189µ -9.04% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
│ old.txt │ new.txt │
│ items/sec │ items/sec vs base │
_Uint64Map_Random-8 1.644M ± ∞ ¹ 1.697M ± ∞ ¹ +3.22% (p=0.008 n=5)
_Uint64Map_Sequential-8 1.980M ± ∞ ¹ 2.052M ± ∞ ¹ +3.64% (p=0.008 n=5)
_Uint64Map_Sequential_Insert-8 2.131M ± ∞ ¹ 2.228M ± ∞ ¹ +4.52% (p=0.008 n=5)
_Uint64Map_Sequential_Txn_Insert-8 14.20M ± ∞ ¹ 14.31M ± ∞ ¹ ~ (p=0.310 n=5)
_Uint64Map_Random_Insert-8 1.773M ± ∞ ¹ 1.870M ± ∞ ¹ +5.45% (p=0.008 n=5)
_Uint64Map_Random_Txn_Insert-8 7.661M ± ∞ ¹ 9.165M ± ∞ ¹ +19.64% (p=0.008 n=5)
geomean 3.320M 3.520M +6.04%
¹ need >= 6 samples for confidence interval at level 0.95
│ old.txt │ new.txt │
│ B/op │ B/op vs base │
_Uint64Map_Random-8 2.450Mi ± ∞ ¹ 2.455Mi ± ∞ ¹ +0.19% (p=0.008 n=5)
_Uint64Map_Sequential-8 2.161Mi ± ∞ ¹ 2.160Mi ± ∞ ¹ -0.07% (p=0.008 n=5)
_Uint64Map_Sequential_Insert-8 2.154Mi ± ∞ ¹ 2.152Mi ± ∞ ¹ -0.07% (p=0.008 n=5)
_Uint64Map_Sequential_Txn_Insert-8 87.38Ki ± ∞ ¹ 84.40Ki ± ∞ ¹ -3.41% (p=0.008 n=5)
_Uint64Map_Random_Insert-8 2.442Mi ± ∞ ¹ 2.449Mi ± ∞ ¹ +0.30% (p=0.008 n=5)
_Uint64Map_Random_Txn_Insert-8 173.2Ki ± ∞ ¹ 116.1Ki ± ∞ ¹ -32.94% (p=0.008 n=5)
_Insert_RootOnlyWatch-8 73.00Ki ± ∞ ¹ 85.60Ki ± ∞ ¹ +17.27% (p=0.008 n=5)
_Insert-8 187.1Ki ± ∞ ¹ 198.3Ki ± ∞ ¹ +6.00% (p=0.008 n=5)
_Modify-8 58.38Ki ± ∞ ¹ 72.53Ki ± ∞ ¹ +24.25% (p=0.008 n=5)
_GetInsert-8 58.36Ki ± ∞ ¹ 72.53Ki ± ∞ ¹ +24.28% (p=0.008 n=5)
_Replace-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Replace_RootOnlyWatch-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_txn_1-8 216.0 ± ∞ ¹ 232.0 ± ∞ ¹ +7.41% (p=0.008 n=5)
_txn_10-8 89.00 ± ∞ ¹ 107.00 ± ∞ ¹ +20.22% (p=0.008 n=5)
_txn_100-8 80.00 ± ∞ ¹ 96.00 ± ∞ ¹ +20.00% (p=0.008 n=5)
_txn_1000-8 67.00 ± ∞ ¹ 81.00 ± ∞ ¹ +20.90% (p=0.008 n=5)
_txn_delete_1-8 2952.0 ± ∞ ¹ 648.0 ± ∞ ¹ -78.05% (p=0.008 n=5)
_txn_delete_10-8 333.0 ± ∞ ¹ 106.0 ± ∞ ¹ -68.17% (p=0.008 n=5)
_txn_delete_100-8 70.00 ± ∞ ¹ 47.00 ± ∞ ¹ -32.86% (p=0.008 n=5)
_txn_delete_1000-8 27.00 ± ∞ ¹ 24.00 ± ∞ ¹ -11.11% (p=0.008 n=5)
_Get-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_All-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Iterator_All-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Iterator_Next-8 896.0 ± ∞ ¹ 896.0 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Hashmap_Insert-8 72.52Ki ± ∞ ¹ 72.52Ki ± ∞ ¹ ~ (p=1.000 n=5) ²
_Hashmap_Get_Uint64-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Hashmap_Get_Bytes-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Delete_Random-8 83.797Mi ± ∞ ¹ 2.020Mi ± ∞ ¹ -97.59% (p=0.008 n=5)
_nodeMutatedClear-8 0.000 ± ∞ ¹
_nodeMutatedExists-8 0.000 ± ∞ ¹
_find16-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_findIndex16-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_find4-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_findIndex4-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
geomean ³ -17.24% ⁴ ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean
⁴ benchmark set differs from baseline; geomeans may not be comparable
│ old.txt │ new.txt │
│ allocs/op │ allocs/op vs base │
_Uint64Map_Random-8 7.043k ± ∞ ¹ 7.037k ± ∞ ¹ ~ (p=0.516 n=5)
_Uint64Map_Sequential-8 6.754k ± ∞ ¹ 6.753k ± ∞ ¹ -0.01% (p=0.008 n=5)
_Uint64Map_Sequential_Insert-8 5.753k ± ∞ ¹ 5.752k ± ∞ ¹ -0.02% (p=0.008 n=5)
_Uint64Map_Sequential_Txn_Insert-8 2.039k ± ∞ ¹ 2.029k ± ∞ ¹ -0.49% (p=0.008 n=5)
_Uint64Map_Random_Insert-8 6.037k ± ∞ ¹ 6.038k ± ∞ ¹ ~ (p=0.325 n=5)
_Uint64Map_Random_Txn_Insert-8 2.854k ± ∞ ¹ 2.409k ± ∞ ¹ -15.59% (p=0.008 n=5)
_Insert_RootOnlyWatch-8 2.045k ± ∞ ¹ 2.034k ± ∞ ¹ -0.54% (p=0.008 n=5)
_Insert-8 3.082k ± ∞ ¹ 3.061k ± ∞ ¹ -0.68% (p=0.008 n=5)
_Modify-8 1.015k ± ∞ ¹ 1.008k ± ∞ ¹ -0.69% (p=0.008 n=5)
_GetInsert-8 1.015k ± ∞ ¹ 1.008k ± ∞ ¹ -0.69% (p=0.008 n=5)
_Replace-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Replace_RootOnlyWatch-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_txn_1-8 4.000 ± ∞ ¹ 4.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_txn_10-8 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_txn_100-8 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_txn_1000-8 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_txn_delete_1-8 5.000 ± ∞ ¹ 4.000 ± ∞ ¹ -20.00% (p=0.008 n=5)
_txn_delete_10-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_txn_delete_100-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_txn_delete_1000-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Get-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_All-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Iterator_All-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Iterator_Next-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Hashmap_Insert-8 20.00 ± ∞ ¹ 20.00 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Hashmap_Get_Uint64-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Hashmap_Get_Bytes-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_Delete_Random-8 146.5k ± ∞ ¹ 102.4k ± ∞ ¹ -30.12% (p=0.008 n=5)
_nodeMutatedClear-8 0.000 ± ∞ ¹
_nodeMutatedExists-8 0.000 ± ∞ ¹
_find16-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_findIndex16-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_find4-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
_findIndex4-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
geomean ³ -2.42% ⁴ ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean
⁴ benchmark set differs from baseline; geomeans may not be comparable
│ old.txt │ new.txt │
│ objects/sec │ objects/sec vs base │
_Insert_RootOnlyWatch-8 13.65M ± ∞ ¹ 13.74M ± ∞ ¹ ~ (p=0.310 n=5)
_Insert-8 9.854M ± ∞ ¹ 9.562M ± ∞ ¹ -2.96% (p=0.008 n=5)
_Modify-8 16.83M ± ∞ ¹ 16.17M ± ∞ ¹ -3.91% (p=0.008 n=5)
_GetInsert-8 13.01M ± ∞ ¹ 12.86M ± ∞ ¹ -1.10% (p=0.032 n=5)
_Replace-8 50.28M ± ∞ ¹ 47.16M ± ∞ ¹ -6.21% (p=0.008 n=5)
_Replace_RootOnlyWatch-8 50.26M ± ∞ ¹ 47.16M ± ∞ ¹ -6.16% (p=0.008 n=5)
_txn_1-8 6.695M ± ∞ ¹ 7.753M ± ∞ ¹ +15.80% (p=0.008 n=5)
_txn_10-8 10.97M ± ∞ ¹ 13.47M ± ∞ ¹ +22.77% (p=0.008 n=5)
_txn_100-8 15.70M ± ∞ ¹ 15.66M ± ∞ ¹ ~ (p=1.000 n=5)
_txn_1000-8 13.54M ± ∞ ¹ 13.70M ± ∞ ¹ ~ (p=0.056 n=5)
_txn_delete_1-8 2.106M ± ∞ ¹ 5.550M ± ∞ ¹ +163.52% (p=0.008 n=5)
_txn_delete_10-8 10.31M ± ∞ ¹ 14.46M ± ∞ ¹ +40.22% (p=0.008 n=5)
_txn_delete_100-8 15.79M ± ∞ ¹ 16.42M ± ∞ ¹ +4.02% (p=0.008 n=5)
_txn_delete_1000-8 18.84M ± ∞ ¹ 19.93M ± ∞ ¹ +5.77% (p=0.008 n=5)
_Get-8 61.32M ± ∞ ¹ 59.17M ± ∞ ¹ -3.50% (p=0.008 n=5)
_All-8 214.9M ± ∞ ¹ 219.7M ± ∞ ¹ ~ (p=0.056 n=5)
_Iterator_All-8 218.4M ± ∞ ¹ 219.2M ± ∞ ¹ ~ (p=0.841 n=5)
_Iterator_Next-8 206.4M ± ∞ ¹ 202.6M ± ∞ ¹ -1.81% (p=0.008 n=5)
_Hashmap_Insert-8 26.50M ± ∞ ¹ 25.29M ± ∞ ¹ -4.57% (p=0.008 n=5)
_Hashmap_Get_Uint64-8 175.7M ± ∞ ¹ 174.4M ± ∞ ¹ ~ (p=1.000 n=5)
_Hashmap_Get_Bytes-8 163.6M ± ∞ ¹ 158.0M ± ∞ ¹ -3.42% (p=0.032 n=5)
_Delete_Random-8 2.491M ± ∞ ¹ 8.534M ± ∞ ¹ +242.51% (p=0.008 n=5)
geomean 25.46M 28.75M +12.93%
¹ need >= 6 samples for confidence interval at level 0.95
Signed-off-by: Jussi Maki <jussi.maki@isovalent.com>
7b19b7c to
4d4ef5c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add
txnIDto each inner node struct. This allows checking if a node had been cloned during this transaction or not which is more efficient and accurate than with nodeMutated. For iterating queries against the tree thetxnIDis bumped so that in-placed mutations are not allowed (much faster than cleaningnodeMutated).The 'NoCache' option was removed as there was no longer any benefit with it (it was used earlier with Set[T] where it was known that there was always just one write per transaction and no point with nodeMutated record keeping).
Deletion now short-circuits if it hits a parent that had already been cloned and we can stop updating the parent pointers.
Benchmarks:
Memory usage actually slightly improved in reconciler benchmark when
txnIDis only in the inner nodes. Having it also on leafs showed marked increase in memory usage, so decision was to only have it on inner nodes.