Make rebalancer prefer buckets without rw refs #633
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently one part of the bucket-related code is at the top of file, another one - in the middle. The current commit is called upon to solve this "problem". The commit doesn`t make any changes to the code, just moves it. The diff is ugly, since the default git algorithm matches poorly here, use `git diff --patience` instead. The main motivation is to move the ref-related code before bucket_transfer_start/end(), which will use refs in the upcoming commits. The `check_is_master()` function is moved to the helper section, where it belongs, since it's used by the ref code. Needed for tarantool#351 NO_DOC=refactoring NO_TEST=refactoring
Serpentian
assigned
kamenkremen,
mrForza,
Gerold103 and
Serpentian and unassigned
Gerold103,
kamenkremen and
mrForza
Serpentian
force-pushed
the
gh-351-reblancer-takes-buckets-with-no-refs
branch
from
efe8655 to
15c2a62
Compare
Collaborator
Author
|
Serpentian
force-pushed
the
gh-351-reblancer-takes-buckets-with-no-refs
branch
3 times, most recently
from
a84098a to
5ca27b9
Compare
This commit adds new functions, which are meant to be used instead of the bucket_transfer_start/end() when sending a bucket. This is needed for tarantool#351, where rebalancer will start to prefer the buckets without refs. For that to work we need to mark the bucket as `transfering` and set `rw_lock` far earlier, than `bucket_send` happens(). The first one is required in order to simplify the process of picking buckets and helps not to pick the same one twice. The second one is essential, since we don't want new refs to happen, when we picked the bucket for sending. The explicit check for being master is dropped from the `bucket_send()` function, since it was never needed there, the check is done in the `bucket_refrw_touch()` anyway. The commit also encapsulates the pcalling of `bucket_send_xc` to separate function: `bucket_send_perform()`, which will also be needed for applier of routes to send a bucket. Part of tarantool#351 NO_DOC=refactoring NO_TEST=refactoring
The `rebalancer_in_work` test case enabled the rebalancer even when it was already enabled. Moreover, at the end of the test the rebalancer is disabled, consequently it doesn't work for the subsequent tests, which is strange for the test, which exists exclusively to test the rebalancer. The rebalancer itself isn't needed for existing tests, but it will be required for the tests, added in the next commit, so let's drop enabling/disabling the rebalancer from the test. Needed for tarantool#351 NO_DOC=test
Before this commit the rebalancer picked the first encountered bucket and waited for number of rw refs on it to become 0. This doesn't work reliably, when cluster has very long RW requests, which block bucket rebalancing for hours: the instance picks the same buckets and timeouts, while waiting for no refs. In order to fix that let's prefer buckets without refs. If there's not enough such buckets, take the remainings from the first available buckets, as it worked before. From now on rebalancer routes applier knows all of the buckets, which are going to be sent before these sends start, which is required for fixing tarantool#573 and tarantool#333. Needed for tarantool#333 Needed for tarantool#573 Closes tarantool#351 NO_DOC=bugfix
After the previuos commit the test became pretty flaky and fails with error "Duplicate key exists in unique index 'pk' in space '_bucket'", when trying to force create buckets. Let's properly wait for all buckets to be garbage collected and not just one, as it was before. Follow-up tarantool#351 NO_DOC=test
Serpentian
force-pushed
the
gh-351-reblancer-takes-buckets-with-no-refs
branch
from
5ca27b9 to
744a3fe
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Disclaimer
Dear reviewers, please don't be afraid by the patch size, I'm just moving the code)
Closes #351