Skip to content

Conversation

@dgpp-programer
Copy link

When I upgraded RocksDB from version v5.10.1 to v8.11.4, a compaction error occurred:
2025/12/19-08:41:47.105378 139671781758720 [WARN] [db/compaction/compaction_job.cc:848] [default] [JOB 6] Compaction Total number of input records: 44912, but processed 44911 records.
The root cause of this issue is that the SST metadata in RocksDB v5.10.3 does not include the num_range_deletion field, whereas this field was added in v8.11.4 with a default value of 0.
After the upgrade, when compaction reads the metadata of legacy SST files, it fails to retrieve the num_range_deletions value and thus assigns it a default of 0. However, these legacy SST files actually contain DeleteRange operations.
Upon completion of compaction, a validation check is performed:
// rocksdb-8.11.4/db/compaction/compaction_job.cc uint64_t expected = compaction_stats_.stats.num_input_records - num_input_range_del; uint64_t actual = compaction_job_stats_->num_input_records; if (expected != actual) { xxx // error
Since num_range_deletion is assigned a value of 0, this validation check fails.

Reproduction code as follows:
Generate data directory with v5.10.3:
int main() { Options options; options.create_if_missing = true; DB* db; Status s = DB::Open(options, "../db", &db); assert(s.ok()); auto generate_sst = [&]() { for (int i = 0; i < 100; ++i) { string key = "key" + to_string(i); string value = "value" + to_string(i); s = db->Put(WriteOptions(), key, value); assert(s.ok()); } s = db->DeleteRange(WriteOptions(), db->DefaultColumnFamily(), "key3", "key8"); assert(s.ok()); s = db->Flush(FlushOptions()); assert(s.ok()); }; // generate 2 sst files generate_sst(); generate_sst(); delete db; return 0; }
Upgrade and verify with v8.11.4:
int main() { DB* db; Options options; options.create_if_missing = true; Status status = DB::Open(options, "../db/", &db); if (!status.ok()) { std::cout << "open db failed: " << status.ToString() << std::endl; return -1; } status = db->CompactRange(CompactRangeOptions(), nullptr, nullptr); if (!status.ok()) { std::cout << "compact db failed: " << status.ToString() << std::endl; return -1; } delete db; return 0; }
// will print
compact db failed: Corruption: Compaction number of input keys does not match number of keys processed. Expected 92 but processed 90. Compaction summary: Base version 2 Base level 0, inputs: [10(1881B) 7(1881B)]

The fix approach is as follows: when num_range_deletions is 0, check whether there are fragmented_range_dels. If fragmented_range_dels exist, throw an error and skip the validation check following the original process.

@meta-cla meta-cla bot added the CLA Signed label Jan 15, 2026
@dgpp-programer
Copy link
Author

Hi maintainers, could you please review and approve the workflow for this PR? Thanks! @ajkr @pdillinger

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant