feat: impose bound on number of batches finding quorum #20
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There is no mechanism as we speak to expire batches that were never able to find quorum. This will eventually lead to storage becoming unmanageably large, and bad actors (if they had permission to) could attack this unbounded storage with submissions that will intentionally never reach quorum. A basic protection against this would be to prune batches that are still finding quorum after a certain limit has been reached. This change accomplishes this, keeping track of batch submission order with a new BatchQueue and attempting to prune submissions from the back of the queue once a configurable capacity is reached.
Recently, the indexing pallet imposed a bound on batches finding quorum with the help of a new BatchQueue in storage. This adds a storage migration to populate the initial contents of this BatchQueue, by analyzing the submissions storage. This implementation is simple but dumb. There's no way for us to know what order the batches were initially submitted in from current storage, so this just inserts them to the batch queue in their iteration order (lexicographic ordering of the hash of the batch ids). Furthermore, this does not do any pruning, so ideally this the batch queue capacity should exceed the number of batches finding quorum in all live networks so that they are pruned more gradually.
tlovell-sxt
force-pushed
the
feat/bound-submissions
branch
from
f428573 to
cb30828
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
There is no mechanism as we speak to expire batches that were never able to find quorum. This will eventually lead to storage becoming unmanageably large, and bad actors (if they had permission to) could attack this unbounded storage with submissions that will intentionally never reach quorum. A basic protection against this would be to prune batches that are still finding quorum after a certain limit has been reached. This change accomplishes this, keeping track of batch submission order with a new BatchQueue and attempting to prune submissions from the back of the queue once a configurable capacity is reached.
What changes are included in this PR?
The commits provide a decent summary:
Are these changes tested?
Yes.