From 8ce635de4569433ba3c80b37817ae50fc00340a7 Mon Sep 17 00:00:00 2001 From: Wenqi Mou Date: Tue, 3 Jun 2025 15:10:53 -0400 Subject: [PATCH 01/11] br: pitr filter feature release doc Signed-off-by: Wenqi Mou --- br/br-pitr-manual.md | 96 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 88 insertions(+), 8 deletions(-) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index b3f007c8976a1..78394edcd6d14 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -496,6 +496,84 @@ tiup br restore point --pd="${PD_IP}:2379" --master-key "local:///path/to/master.key" ``` +### Restore with filters + +Starting from TiDB v9.0.0, you can use filters during PITR to selectively restore specific databases or tables. This allows for more granular control over what data gets restored during point-in-time recovery operations. + + +The filter patterns follow the same syntax as [table filters](/table-filter.md) used in other BR operations: + +- `'*.*'` - matches all databases and tables +- `'db1.*'` - matches all tables in database `db1` +- `'db1.table1'` - matches specific table `table1` in database `db1` +- `'db*.tbl*'` - matches databases starting with `db` and tables starting with `tbl` +- `'!mysql.*'` - excludes all tables in the `mysql` database + +Usage examples: + +```shell +# restore specific databases +tiup br restore point --pd="${PD_IP}:2379" \ +--storage='s3://backup-101/logbackup?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--full-backup-storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--start-ts "2025-06-02 00:00:00+0800" \ +--restored-ts "2025-06-03 18:00:00+0800" \ +--filter 'db1.*' --filter 'db2.*' + +# restore specific tables +tiup br restore point --pd="${PD_IP}:2379" \ +--storage='s3://backup-101/logbackup?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--full-backup-storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--start-ts "2025-06-02 00:00:00+0800" \ +--restored-ts "2025-06-03 18:00:00+0800" \ +--filter 'db1.users' --filter 'db1.orders' + +# restore with pattern matching +tiup br restore point --pd="${PD_IP}:2379" \ +--storage='s3://backup-101/logbackup?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--full-backup-storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--start-ts "2025-06-02 00:00:00+0800" \ +--restored-ts "2025-06-03 18:00:00+0800" \ +--filter 'db*.tbl*' +``` + +> **Note:** +> +> - When using filters, ensure that the filtered data maintains referential integrity. +> - Filter options apply to both snapshot and log backup restoration phases. +> - Multiple `--filter` options can be specified to include or exclude different patterns. +> - PITR filtering does not support restoring system tables. System tables are restored according to the full backup data and cannot be selectively filtered during PITR operations. If you need to restore specific system tables, use `br restore full` with filters instead, which will restore only the snapshot backup data (not log backup data). +> - The target databases and tables specified in the filter must not exist in the cluster, or the restore will fail with an error. + +### Concurrent restore operations + +Starting from TiDB v9.0.0, you can run multiple PITR operations concurrently. This feature allows you to restore different datasets simultaneously, improving restore efficiency for large-scale operations. + +Usage example for concurrent restores: + +```shell +# terminal 1 - restore database db1 +tiup br restore point --pd="${PD_IP}:2379" \ +--storage='s3://backup-101/logbackup?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--full-backup-storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--start-ts "2025-06-02 00:00:00+0800" \ +--restored-ts "2025-06-03 18:00:00+0800" \ +--filter 'db1.*' + +# terminal 2 - restore database db2 (can run simultaneously) +tiup br restore point --pd="${PD_IP}:2379" \ +--storage='s3://backup-101/logbackup?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--full-backup-storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--start-ts "2025-06-02 00:00:00+0800" \ +--restored-ts "2025-06-03 18:00:00+0800" \ +--filter 'db2.*' +``` + +> **Note:** +> +> - Each concurrent restore operation must target different databases or non-overlapping table sets. Attempting to restore overlapping datasets concurrently will result in an error. + + ### Compatibility between ongoing log backup and snapshot restore Starting from v9.0.0, when a log backup task is running, if all of the following conditions are met, you can still perform snapshot restore (`br restore [full|database|table]`) and allow the restored data to be properly recorded by the ongoing log backup (hereinafter referred to as "log backup"): @@ -507,13 +585,15 @@ Starting from v9.0.0, when a log backup task is running, if all of the following - The data to be restored uses the same type of external storage as the target storage for the log backup. - Neither the data to be restored nor the log backup has enabled local encryption. For details, see [log backup encryption](#encrypt-the-log-backup-data) and [snapshot backup encryption](/br/br-snapshot-manual.md#encrypt-the-backup-data). -If any of the above conditions are not met, or if you need to perform a point-in-time recovery, while a log backup task is running, BR refuses to proceed with the data recovery. In this case, you can complete the recovery by following these steps: +### Compatibility between ongoing log backup and PITR operations -1. [Stop the log backup task](#stop-a-log-backup-task). -2. Perform the data restore. -3. After the restore is complete, perform a new snapshot backup. -4. [Restart the log backup task](#restart-a-log-backup-task). +Starting from TiDB v9.0.0, you can perform PITR operations while a log backup task is running by default. The system automatically handles compatibility between these operations. -> **Note:** -> -> When restoring a log backup that contains records of snapshot (full) restore data, you must use BR v9.0.0 or later. Otherwise, restoring the recorded full restore data might fail. \ No newline at end of file +#### Important limitation for PITR with ongoing log backup + +When performing PITR operations while log backup is running, the restored data will be recorded in the log backup. However, during the restore time window, the data may not be consistent due to the nature of log restore operations. The system writes metadata to external storage to mark both the time range and data range where consistency cannot be guaranteed. + +If such inconsistency occurs during the time range `[t1, t2)`, you cannot restore data from that time period. Instead, you must: + +- Restore data up to `t1` (before the inconsistent period), or +- Take a new snapshot backup after `t2` and use that for future PITR operations \ No newline at end of file From 0ac24f65db59072794a770e93724c29bb39141d5 Mon Sep 17 00:00:00 2001 From: Wenqi Mou Date: Tue, 3 Jun 2025 15:15:19 -0400 Subject: [PATCH 02/11] fix Signed-off-by: Wenqi Mou --- br/br-pitr-manual.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index 78394edcd6d14..d80b622891ab8 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -585,6 +585,16 @@ Starting from v9.0.0, when a log backup task is running, if all of the following - The data to be restored uses the same type of external storage as the target storage for the log backup. - Neither the data to be restored nor the log backup has enabled local encryption. For details, see [log backup encryption](#encrypt-the-log-backup-data) and [snapshot backup encryption](/br/br-snapshot-manual.md#encrypt-the-backup-data). +If any of the above conditions are not met, you can complete the recovery by following these steps: +1. [Stop the log backup task](#stop-a-log-backup-task). +2. Perform the data restore. +3. After the restore is complete, perform a new snapshot backup. +4. [Restart the log backup task](#restart-a-log-backup-task). + +> **Note:** +> +> When restoring a log backup that contains records of snapshot (full) restore data, you must use BR v9.0.0 or later. Otherwise, restoring the recorded full restore data might fail. + ### Compatibility between ongoing log backup and PITR operations Starting from TiDB v9.0.0, you can perform PITR operations while a log backup task is running by default. The system automatically handles compatibility between these operations. From b14ce78451d9f946d030ee20a8ebea4d735a65d0 Mon Sep 17 00:00:00 2001 From: Wenqi Mou Date: Tue, 3 Jun 2025 15:24:14 -0400 Subject: [PATCH 03/11] Update br-pitr-manual.md --- br/br-pitr-manual.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index d80b622891ab8..a49c5f19aaf37 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -500,7 +500,6 @@ tiup br restore point --pd="${PD_IP}:2379" Starting from TiDB v9.0.0, you can use filters during PITR to selectively restore specific databases or tables. This allows for more granular control over what data gets restored during point-in-time recovery operations. - The filter patterns follow the same syntax as [table filters](/table-filter.md) used in other BR operations: - `'*.*'` - matches all databases and tables @@ -573,7 +572,6 @@ tiup br restore point --pd="${PD_IP}:2379" \ > > - Each concurrent restore operation must target different databases or non-overlapping table sets. Attempting to restore overlapping datasets concurrently will result in an error. - ### Compatibility between ongoing log backup and snapshot restore Starting from v9.0.0, when a log backup task is running, if all of the following conditions are met, you can still perform snapshot restore (`br restore [full|database|table]`) and allow the restored data to be properly recorded by the ongoing log backup (hereinafter referred to as "log backup"): @@ -606,4 +604,5 @@ When performing PITR operations while log backup is running, the restored data w If such inconsistency occurs during the time range `[t1, t2)`, you cannot restore data from that time period. Instead, you must: - Restore data up to `t1` (before the inconsistent period), or -- Take a new snapshot backup after `t2` and use that for future PITR operations \ No newline at end of file +- Take a new snapshot backup after `t2` and use that for future PITR operations +- From 00139ff948f48474fe41203331af9774d7753b3c Mon Sep 17 00:00:00 2001 From: Wenqi Mou Date: Tue, 3 Jun 2025 15:40:20 -0400 Subject: [PATCH 04/11] update Signed-off-by: Wenqi Mou --- br/br-pitr-manual.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index a49c5f19aaf37..2df98056c9163 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -541,7 +541,7 @@ tiup br restore point --pd="${PD_IP}:2379" \ > - When using filters, ensure that the filtered data maintains referential integrity. > - Filter options apply to both snapshot and log backup restoration phases. > - Multiple `--filter` options can be specified to include or exclude different patterns. -> - PITR filtering does not support restoring system tables. System tables are restored according to the full backup data and cannot be selectively filtered during PITR operations. If you need to restore specific system tables, use `br restore full` with filters instead, which will restore only the snapshot backup data (not log backup data). +> - PITR filtering does not support system tables yet. If you need to restore specific system tables, use `br restore full` with filters instead, which will restore only the snapshot backup data (not log backup data). > - The target databases and tables specified in the filter must not exist in the cluster, or the restore will fail with an error. ### Concurrent restore operations From 5c0c9b3c8a9e64c624a072b91519d0a49cf05f12 Mon Sep 17 00:00:00 2001 From: Wenqi Mou Date: Thu, 26 Jun 2025 20:27:28 -0400 Subject: [PATCH 05/11] Apply suggestions from code review Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- br/br-pitr-manual.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index 2df98056c9163..fe1d773dd0c3e 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -538,7 +538,7 @@ tiup br restore point --pd="${PD_IP}:2379" \ > **Note:** > -> - When using filters, ensure that the filtered data maintains referential integrity. +When you use filters, ensure that the filtered data maintains referential integrity. > - Filter options apply to both snapshot and log backup restoration phases. > - Multiple `--filter` options can be specified to include or exclude different patterns. > - PITR filtering does not support system tables yet. If you need to restore specific system tables, use `br restore full` with filters instead, which will restore only the snapshot backup data (not log backup data). @@ -599,7 +599,7 @@ Starting from TiDB v9.0.0, you can perform PITR operations while a log backup ta #### Important limitation for PITR with ongoing log backup -When performing PITR operations while log backup is running, the restored data will be recorded in the log backup. However, during the restore time window, the data may not be consistent due to the nature of log restore operations. The system writes metadata to external storage to mark both the time range and data range where consistency cannot be guaranteed. +When you perform PITR operations while log backup is running, the restored data will be recorded in the log backup. However, during the restore time window, the data may not be consistent due to the nature of log restore operations. The system writes metadata to external storage to mark both the time range and data range where consistency cannot be guaranteed. If such inconsistency occurs during the time range `[t1, t2)`, you cannot restore data from that time period. Instead, you must: From d898e96e683c956446c315bd36202df5333c098a Mon Sep 17 00:00:00 2001 From: Wenqi Mou Date: Mon, 30 Jun 2025 17:45:20 -0400 Subject: [PATCH 06/11] add abort command Signed-off-by: Wenqi Mou --- br/br-pitr-manual.md | 60 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 47 insertions(+), 13 deletions(-) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index fe1d773dd0c3e..ac0bf8ae257f7 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -502,11 +502,11 @@ Starting from TiDB v9.0.0, you can use filters during PITR to selectively restor The filter patterns follow the same syntax as [table filters](/table-filter.md) used in other BR operations: -- `'*.*'` - matches all databases and tables -- `'db1.*'` - matches all tables in database `db1` -- `'db1.table1'` - matches specific table `table1` in database `db1` -- `'db*.tbl*'` - matches databases starting with `db` and tables starting with `tbl` -- `'!mysql.*'` - excludes all tables in the `mysql` database +- `'*.*'`: matches all databases and tables. +- `'db1.*'`: matches all tables in database `db1`. +- `'db1.table1'`: matches specific table `table1` in database `db1`. +- `'db*.tbl*'`: matches databases starting with `db` and tables starting with `tbl`. +- `'!mysql.*'`: excludes all tables in the `mysql` database. Usage examples: @@ -538,15 +538,14 @@ tiup br restore point --pd="${PD_IP}:2379" \ > **Note:** > -When you use filters, ensure that the filtered data maintains referential integrity. +> - When using filters, ensure that the target cluster does not contain databases or tables that match the filter, or the restore will fail with an error. > - Filter options apply to both snapshot and log backup restoration phases. > - Multiple `--filter` options can be specified to include or exclude different patterns. > - PITR filtering does not support system tables yet. If you need to restore specific system tables, use `br restore full` with filters instead, which will restore only the snapshot backup data (not log backup data). -> - The target databases and tables specified in the filter must not exist in the cluster, or the restore will fail with an error. ### Concurrent restore operations -Starting from TiDB v9.0.0, you can run multiple PITR operations concurrently. This feature allows you to restore different datasets simultaneously, improving restore efficiency for large-scale operations. +Starting from TiDB v9.0.0, you can run multiple PITR restore tasks concurrently. This feature allows you to restore different datasets in parallel, improving efficiency for large-scale restore scenarios. Usage example for concurrent restores: @@ -570,7 +569,7 @@ tiup br restore point --pd="${PD_IP}:2379" \ > **Note:** > -> - Each concurrent restore operation must target different databases or non-overlapping table sets. Attempting to restore overlapping datasets concurrently will result in an error. +> Each concurrent restore operation must target different databases or non-overlapping table sets. Attempting to restore overlapping datasets concurrently will result in an error. ### Compatibility between ongoing log backup and snapshot restore @@ -599,10 +598,45 @@ Starting from TiDB v9.0.0, you can perform PITR operations while a log backup ta #### Important limitation for PITR with ongoing log backup -When you perform PITR operations while log backup is running, the restored data will be recorded in the log backup. However, during the restore time window, the data may not be consistent due to the nature of log restore operations. The system writes metadata to external storage to mark both the time range and data range where consistency cannot be guaranteed. +When you perform PITR operations while log backup is running, the restored data will also be recorded in the log backup. However, during the restore operation time window, there may be data inconsistency risks due to the nature of log restore operations. The system writes metadata to external storage to mark both the time range and data range where consistency cannot be guaranteed. -If such inconsistency occurs during the time range `[t1, t2)`, you cannot restore data from that time period. Instead, you must: +If such inconsistency occurs during the time range `[t1, t2)`, you cannot directly restore data from that time period and must choose one of the following alternatives: -- Restore data up to `t1` (before the inconsistent period), or +- Restore data up to `t1` (before the inconsistent period) - Take a new snapshot backup after `t2` and use that for future PITR operations -- + +## Abort restore operations + +You can use the `tiup br abort` command to clean up registry entries and checkpoint data when a restore operation fails. This command automatically finds and removes the relevant metadata based on the original restore parameters, including entries in the `mysql.tidb_restore_registry` table and checkpoint data (whether stored in the local database or external storage). Note that the `abort` command only cleans up metadata; any actual restored data must be manually dropped from the cluster. + +### Usage examples + +Use the same parameters as the original restore command: + +```shell +# Abort a PITR operation +tiup br abort restore point --pd="${PD_IP}:2379" \ +--storage='s3://backup-101/logbackup?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--full-backup-storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' + +# Abort a PITR operation with filters +tiup br abort restore point --pd="${PD_IP}:2379" \ +--storage='s3://backup-101/logbackup?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--full-backup-storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--filter 'db1.*' + +# Abort a full restore +tiup br abort restore full --pd="${PD_IP}:2379" \ +--storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' + +# Abort a database restore +tiup br abort restore db --pd="${PD_IP}:2379" \ +--storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--db database_name + +# Abort a table restore +tiup br abort restore table --pd="${PD_IP}:2379" \ +--storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ +--db database_name --table table_name +``` + From 502479833e0930535b88096d6030fd87a24375b9 Mon Sep 17 00:00:00 2001 From: Wenqi Mou Date: Mon, 30 Jun 2025 21:34:32 -0400 Subject: [PATCH 07/11] Update br-pitr-manual.md --- br/br-pitr-manual.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index ac0bf8ae257f7..2a34c7fe958fc 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -583,6 +583,7 @@ Starting from v9.0.0, when a log backup task is running, if all of the following - Neither the data to be restored nor the log backup has enabled local encryption. For details, see [log backup encryption](#encrypt-the-log-backup-data) and [snapshot backup encryption](/br/br-snapshot-manual.md#encrypt-the-backup-data). If any of the above conditions are not met, you can complete the recovery by following these steps: + 1. [Stop the log backup task](#stop-a-log-backup-task). 2. Perform the data restore. 3. After the restore is complete, perform a new snapshot backup. @@ -639,4 +640,3 @@ tiup br abort restore table --pd="${PD_IP}:2379" \ --storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ --db database_name --table table_name ``` - From eaeee2ff2a61ab3583747a15a7ee52ba9c242b5b Mon Sep 17 00:00:00 2001 From: Wenqi Mou Date: Fri, 11 Jul 2025 09:36:55 -0400 Subject: [PATCH 08/11] Apply suggestions from code review Co-authored-by: xixirangrang --- br/br-pitr-manual.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index 2a34c7fe958fc..765cbbd122e6c 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -503,8 +503,8 @@ Starting from TiDB v9.0.0, you can use filters during PITR to selectively restor The filter patterns follow the same syntax as [table filters](/table-filter.md) used in other BR operations: - `'*.*'`: matches all databases and tables. -- `'db1.*'`: matches all tables in database `db1`. -- `'db1.table1'`: matches specific table `table1` in database `db1`. +- `'db1.*'`: matches all tables in the database `db1`. +- `'db1.table1'`: matches the specific table `table1` in the database `db1`. - `'db*.tbl*'`: matches databases starting with `db` and tables starting with `tbl`. - `'!mysql.*'`: excludes all tables in the `mysql` database. @@ -538,7 +538,7 @@ tiup br restore point --pd="${PD_IP}:2379" \ > **Note:** > -> - When using filters, ensure that the target cluster does not contain databases or tables that match the filter, or the restore will fail with an error. +> - When using filters, ensure that the target cluster does not contain databases or tables that match the filter. Otherwise, the restore will fail with an error. > - Filter options apply to both snapshot and log backup restoration phases. > - Multiple `--filter` options can be specified to include or exclude different patterns. > - PITR filtering does not support system tables yet. If you need to restore specific system tables, use `br restore full` with filters instead, which will restore only the snapshot backup data (not log backup data). @@ -599,7 +599,7 @@ Starting from TiDB v9.0.0, you can perform PITR operations while a log backup ta #### Important limitation for PITR with ongoing log backup -When you perform PITR operations while log backup is running, the restored data will also be recorded in the log backup. However, during the restore operation time window, there may be data inconsistency risks due to the nature of log restore operations. The system writes metadata to external storage to mark both the time range and data range where consistency cannot be guaranteed. +When you perform PITR operations while log backup is running, the restored data will also be recorded in the log backup. However, during the restore operation time window, there are data inconsistency risks due to the nature of log restore operations. The system writes metadata to external storage to mark both the time range and data range where consistency cannot be guaranteed. If such inconsistency occurs during the time range `[t1, t2)`, you cannot directly restore data from that time period and must choose one of the following alternatives: @@ -608,7 +608,7 @@ If such inconsistency occurs during the time range `[t1, t2)`, you cannot direct ## Abort restore operations -You can use the `tiup br abort` command to clean up registry entries and checkpoint data when a restore operation fails. This command automatically finds and removes the relevant metadata based on the original restore parameters, including entries in the `mysql.tidb_restore_registry` table and checkpoint data (whether stored in the local database or external storage). Note that the `abort` command only cleans up metadata; any actual restored data must be manually dropped from the cluster. +You can use the `tiup br abort` command to clean up registry entries and checkpoint data when a restore operation fails. This command automatically finds and removes the relevant metadata based on the original restore parameters, including entries in the `mysql.tidb_restore_registry` table and checkpoint data (whether stored in the local database or external storage). Note that the `abort` command only cleans up metadata. Any actual restored data must be manually dropped from the cluster. ### Usage examples From fabd5a8edafadcdf6663c7561d08ae2c6479371f Mon Sep 17 00:00:00 2001 From: Wenqi Mou Date: Wed, 16 Jul 2025 15:25:18 -0400 Subject: [PATCH 09/11] Apply suggestions from code review Co-authored-by: Lilian Lee --- br/br-pitr-manual.md | 36 ++++++++++++++++++++---------------- 1 file changed, 20 insertions(+), 16 deletions(-) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index 765cbbd122e6c..4a192ad92e360 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -496,11 +496,11 @@ tiup br restore point --pd="${PD_IP}:2379" --master-key "local:///path/to/master.key" ``` -### Restore with filters +### Restore data using filters -Starting from TiDB v9.0.0, you can use filters during PITR to selectively restore specific databases or tables. This allows for more granular control over what data gets restored during point-in-time recovery operations. +Starting from TiDB v9.0.0, you can use filters during PITR to restore specific databases or tables, enabling more fine-grained control over the data to be restored. -The filter patterns follow the same syntax as [table filters](/table-filter.md) used in other BR operations: +The filter patterns follow the same [table filtering syntax](/table-filter.md) as other BR operations: - `'*.*'`: matches all databases and tables. - `'db1.*'`: matches all tables in the database `db1`. @@ -527,7 +527,7 @@ tiup br restore point --pd="${PD_IP}:2379" \ --restored-ts "2025-06-03 18:00:00+0800" \ --filter 'db1.users' --filter 'db1.orders' -# restore with pattern matching +# restore using pattern matching tiup br restore point --pd="${PD_IP}:2379" \ --storage='s3://backup-101/logbackup?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ --full-backup-storage='s3://backup-101/snapshot-20250602000000?access-key=${ACCESS-KEY}&secret-access-key=${SECRET-ACCESS-KEY}' \ @@ -538,10 +538,10 @@ tiup br restore point --pd="${PD_IP}:2379" \ > **Note:** > -> - When using filters, ensure that the target cluster does not contain databases or tables that match the filter. Otherwise, the restore will fail with an error. -> - Filter options apply to both snapshot and log backup restoration phases. -> - Multiple `--filter` options can be specified to include or exclude different patterns. -> - PITR filtering does not support system tables yet. If you need to restore specific system tables, use `br restore full` with filters instead, which will restore only the snapshot backup data (not log backup data). +> - Before restoring data using filters, ensure that the target cluster does not contain any databases or tables that match the filter. Otherwise, the restore will fail with an error. +> - The filter options apply during the restore phase for both snapshot and log backups. +> - You can specify multiple `--filter` options to include or exclude different patterns. +> - PITR filtering does not support system tables yet. If you need to restore specific system tables, use the `br restore full` command with filters instead. Note that this command restores only the snapshot backup data (not log backup data). ### Concurrent restore operations @@ -569,7 +569,7 @@ tiup br restore point --pd="${PD_IP}:2379" \ > **Note:** > -> Each concurrent restore operation must target different databases or non-overlapping table sets. Attempting to restore overlapping datasets concurrently will result in an error. +> Each concurrent restore operation must target a different database or a non-overlapping set of tables. Attempting to restore overlapping datasets concurrently will result in an error. ### Compatibility between ongoing log backup and snapshot restore @@ -582,7 +582,7 @@ Starting from v9.0.0, when a log backup task is running, if all of the following - The data to be restored uses the same type of external storage as the target storage for the log backup. - Neither the data to be restored nor the log backup has enabled local encryption. For details, see [log backup encryption](#encrypt-the-log-backup-data) and [snapshot backup encryption](/br/br-snapshot-manual.md#encrypt-the-backup-data). -If any of the above conditions are not met, you can complete the recovery by following these steps: +If any of the above conditions are not met, you can restore the data by following these steps: 1. [Stop the log backup task](#stop-a-log-backup-task). 2. Perform the data restore. @@ -599,20 +599,24 @@ Starting from TiDB v9.0.0, you can perform PITR operations while a log backup ta #### Important limitation for PITR with ongoing log backup -When you perform PITR operations while log backup is running, the restored data will also be recorded in the log backup. However, during the restore operation time window, there are data inconsistency risks due to the nature of log restore operations. The system writes metadata to external storage to mark both the time range and data range where consistency cannot be guaranteed. +When you perform the PITR operations while a log backup is running, the restored data will also be recorded in the ongoing log backup. However, due to the nature of log restore operations, data inconsistencies might occur within the restore window. The system writes metadata to external storage to mark both the time range and data range where consistency cannot be guaranteed. -If such inconsistency occurs during the time range `[t1, t2)`, you cannot directly restore data from that time period and must choose one of the following alternatives: +If such inconsistency occurs during the time range `[t1, t2)`, you cannot directly restore data from this period. Instead, choose one of the following alternatives: -- Restore data up to `t1` (before the inconsistent period) -- Take a new snapshot backup after `t2` and use that for future PITR operations +- Restore data up to `t1` (to retrieve data before the inconsistent period). +- Perform a new snapshot backup after `t2`, and use it as the base for future PITR operations. ## Abort restore operations -You can use the `tiup br abort` command to clean up registry entries and checkpoint data when a restore operation fails. This command automatically finds and removes the relevant metadata based on the original restore parameters, including entries in the `mysql.tidb_restore_registry` table and checkpoint data (whether stored in the local database or external storage). Note that the `abort` command only cleans up metadata. Any actual restored data must be manually dropped from the cluster. +If a restore operation fails, you can use the `tiup br abort` command to clean up registry entries and checkpoint data. This command automatically locates and removes relevant metadata based on the original restore parameters, including entries in the `mysql.tidb_restore_registry` table and checkpoint data (regardless of whether it is stored in a local database or external storage). + +> **Note:** +> +> The `abort` command only cleans up metadata. You need to manually delete any actual restored data from the cluster. ### Usage examples -Use the same parameters as the original restore command: +Use the same parameters as in the original restore command: ```shell # Abort a PITR operation From 1257b37e62622b164eeaf3a2a4018ad6bf74c910 Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Fri, 15 Aug 2025 14:32:04 +0800 Subject: [PATCH 10/11] Update heading level for consistent sty --- br/br-pitr-manual.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index 4a192ad92e360..24a05505c9821 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -606,7 +606,7 @@ If such inconsistency occurs during the time range `[t1, t2)`, you cannot direct - Restore data up to `t1` (to retrieve data before the inconsistent period). - Perform a new snapshot backup after `t2`, and use it as the base for future PITR operations. -## Abort restore operations +### Abort restore operations If a restore operation fails, you can use the `tiup br abort` command to clean up registry entries and checkpoint data. This command automatically locates and removes relevant metadata based on the original restore parameters, including entries in the `mysql.tidb_restore_registry` table and checkpoint data (regardless of whether it is stored in a local database or external storage). @@ -614,9 +614,7 @@ If a restore operation fails, you can use the `tiup br abort` command to clean u > > The `abort` command only cleans up metadata. You need to manually delete any actual restored data from the cluster. -### Usage examples - -Use the same parameters as in the original restore command: +The usage examples of aborting restore operations using the same parameters as the original restore command are as follows: ```shell # Abort a PITR operation From 66655a3f3265b7460913dd2ce5c91c8102dc686a Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Fri, 15 Aug 2025 14:35:35 +0800 Subject: [PATCH 11/11] Update wording --- br/br-pitr-manual.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index 24a05505c9821..83516d19ed8e0 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -614,7 +614,7 @@ If a restore operation fails, you can use the `tiup br abort` command to clean u > > The `abort` command only cleans up metadata. You need to manually delete any actual restored data from the cluster. -The usage examples of aborting restore operations using the same parameters as the original restore command are as follows: +The examples of aborting restore operations using the same parameters as the original restore command are as follows: ```shell # Abort a PITR operation