disagg: Fine-grained rate limiting for S3 cache downloads (#10637)

[ti-chi-bot]

This is an automated cherry-pick of #10637

What problem does this PR solve?

Issue Number: close #10633, ref #10557

Problem Summary:
Currently, TiFlash Compute’s FileCache downloads data from S3 into local block storage as follows: it obtains an std::istream object via S3’s GetObject API and requests permission from the RateLimiter based on the total size of the entire S3 object before writing (implemented by #10555). Once the rate limiter approves the request, the full object is written to the local block storage in one burst.
Although this approach may keep the average write throughput below the quota threshold—making it appear compliant—the instantaneous write rate during the actual download can temporarily exceed the quota. This burstiness can trigger throttling by the underlying block storage, leading to elevated I/O latency.

What is changed and how it works?

disagg: Fine-grained rate limiting for S3 cache downloads
  * Use 128KiB buffer for S3 cache downloads and rate limiting. Avoid burst IO that cause block storage throttling

Cloud vendors usually consider 16KiB as 1 IOPS

• AWS: https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volume-types.html
• Alibaba Cloud: https://www.alibabacloud.com/help/en/ecs/user-guide/essd-autopl-disks

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)

1. deploy a next-gen cluster
2. load chbenchmark-1500 dataset and add tiflash replica
go-tpc tpcc prepare --warehouses 1500 -T 30 -D chbenchmark -H 172.31.7.2 -P 4000
go-tpc ch prepare -H 172.31.7.2 -P 4000 -D chbenchmark1k5_clustered 
# deploy 2 tiflash-compute with limited resources and ebs bandwidth
# m7a.2xlarge (8c32G), IOPS 6000 Throughput 125
# patch 1 tiflash-compute with the master
tiup cluster patch ng ./tiflash-patch.965cc8.tar.gz -N 172.31.10.1:9000 -y
# patch 1 tiflash-compute with this branch
tiup cluster patch ng ./tiflash-patch.8096c4.tar.gz -N 172.31.10.2:9000 -y
# shutdown 172.31.10.2 and run chbenchmark on master branch tiflash-compte
go-tpc ch run -H 172.31.7.2 -P4000 --warehouses 1500 -D chbenchmark1k5_clustered -T 50 -t 2 --time 15m
# shutdown 172.31.10.1 and start 172.31.10.2 to run chbenchmark on this branch
go-tpc ch run -H 172.31.7.2 -P4000 --warehouses 1500 -D chbenchmark1k5_clustered -T 50 -t 2 --time 15m
# check the metrics

Performance changes

For the master branch, the write latency will increased to 21ms After improvement by this PR, the write latency and write throughput is more smooth. Max write latency is is 10ms, most of the time is only 2ms.

the read latency is also slightly reduced

# master branch
tpmC: 4218.7, tpmTotal: 9365.9, efficiency: 21.9%
[Summary] Q1     - Count: 133, Sum(ms): 4574862.2, Avg(ms): 34396.9
# this PR
tpmC: 4486.4, tpmTotal: 9974.4, efficiency: 23.3%
[Summary] Q1     - Count: 133, Sum(ms): 4547837.2, Avg(ms): 34194.5

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

Summary by CodeRabbit

• Bug Fixes

  • Safer S3 download flow using temp-file-then-rename to avoid partial/corrupt files and preserve capacity accounting.

• Improvements

  • Consolidated download streaming into a single write-limited local path, improved timing precision, INFO-level logs, and updated download metrics.
  • Added debug-mode size sanity checks and clearer segment state transitions on failure/success.

• Tests

  • Expanded eviction and background-download tests with additional scenarios, assertions, and instrumentation.

• Public API

  • Renamed segment completion setter and added a status getter for clearer segment state handling.

• Chores

  • Updated .gitignore to ignore a new secret file.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[ti-chi-bot]

This cherry pick PR is for a release branch and has not yet been approved by triage owners.
Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick:

1. It must be LGTMed and approved by the reviewers firstly.
2. For pull requests to TiDB-x branches, it must have no failed tests.
3. AFTER it has lgtm and approved labels, please wait for the cherry-pick merging approval from triage owners.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (3)

• release-8.5
• release-7.5
• release-8.1

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign ilovesoup for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment