Skip to content

Commit b79fb5e

Browse files
committed
corrected README and added tts-backup-env-sample
corrected README and added tts-backup-env-sample
1 parent 46ad428 commit b79fb5e

File tree

2 files changed

+122
-90
lines changed

2 files changed

+122
-90
lines changed
Lines changed: 67 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -1,76 +1,44 @@
11
# Transportable Tablespaces Using Backup
22

3-
Transportable Tablespaces can be used to move tablespaces from customer
4-
on-premise or another database cloud service into ADB-S. Tablespaces can
5-
be transported when creating a new database in ADB-S or as modify
6-
operation on an existing database.
3+
Transportable Tablespaces feature can be used to migrate Oracle database tablespaces from customer on-premise or another Oracle Database Cloud Service into Oracle Autonomous AI Database Cloud Service. User tablespaces along with the necessary schemas can be migrated using the Transportable Tablespaces mechanism. Customers can migrate both bigfile and smallfile tabalespaces. Tablespaces be can encrypted or unencrypted. Migration can be done in full or incremental modes. Customers can use Oracle Cloud Infrastructure (OCI) Object Storage or File Storage Service (FSS) as intermediary for the migration. Source database can be of version 11g or higher. Tablespaces can be migrated to Oracle Autonomous AI Database version 19c or 23ai.
74

85
## Step-by-step guide
96

10-
Transporting Tablespaces involves the following steps.
7+
Transporting Tablespaces involves below high level steps.
118

12-
1. Create Object Storage Buckets
13-
14-
2. Create Dynamic Group and Policy
15-
16-
3. Backup Tablespaces on Source Database
17-
18-
4. Create or Modify database in ADB-S by specifying intent to transport
19-
tablespaces using a tag
9+
1. Create OCI Object Storage Buckets or FSS Mount Targets for backup
10+
2. Backup tablespaces on source database using Oracle-provided Backup Utility.
11+
3. Create Dynamic Group and Policy to allow Resource Principal access if using OCI Object Storage
12+
4. Provision an Autonomous AI Database by providing migration inputs.
2013

2114
### Create Object Storage Buckets
2215

23-
Transportable Tablespaces needs two object storage buckets - one for
24-
backups and another for metadata. Create the buckets in your Oracle
25-
Storage Cloud Service account. Note the URLs of the buckets as they are
26-
needed as inputs for the operation. Use [Oracle Cloud Infrastructure Object Storage Native URI Format](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/file-uri-formats.html) for the URL.
16+
Transportable Tablespaces needs two OCI object storage buckets - one for backups and another for metadata. Create the buckets in your Oracle Storage Cloud Service account. Note the URLs of the buckets as they are needed as inputs for the operation. Use [Oracle Cloud Infrastructure Object Storage Native URI Format](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/file-uri-formats.html) for the URL.
2717

2818
Example:
2919
[https://objectstorage.region.oraclecloud.com/n/\<namespace-string\>/b/\<bucket-name\>](https://objectstorage.region.oraclecloud.com/n/namespace-string/b/bucket/o/filename)
3020

31-
To make Object Storage URI work, please [generate an API signing key](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm)].
32-
Download the private key .pem file and the API signing key config file with below contents to source database host.
33-
21+
To make Object Storage URI work, please [generate an API signing key](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm). Download the private key .pem file and the API signing key config file with below contents to source database host.
22+
```
3423
[DEFAULT]
3524
3625
user=ocid1.user.oc1..xxxxx
37-
3826
fingerprint=f6:d6:e5:xxxxx
39-
4027
tenancy=ocid1.tenancy.oc1..xxxxx
41-
4228
region=us-ashburn-1
43-
4429
key_file=<path to the downloaded private key file>
4530
31+
```
4632
Note - User should have read and write access to the object storage buckets.
4733

48-
## Create Dynamic Group and Policy
49-
50-
Transportable Tablespaces functionality will download metadata from metadata bucket using [OCI Resource Principal](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/resource-principal.html)].
51-
52-
### Create Dynamic Group and Policy to allow access to the metadata bucket using the resource principal.
53-
54-
1. Create a Dynamic Group **TTSDynamicGroup** with matching rule:\
55-
ALL {resource.type = \'autonomousdatabase\'[resource.compartment.id](http://resource.compartment.id) = \'your_Compartment_OCID\'}
56-
57-
2. Create a Policy using the dynamic group with Policy Statement:\
58-
Allow dynamic-group **TTSDynamicGroup** to manage buckets in tenancy\
59-
Allow dynamic-group **TTSDynamicGroup** to manage objects in tenancy\
60-
\
61-
Prepend domain name to the dynamic group name if needed as below.\
62-
Allow dynamic-group \<*identity_domain_name\>*/**TTSDynamicGroup** to manage buckets in tenancy\
63-
Allow dynamic-group \<*identity_domain_name\>*/**TTSDynamicGroup** to manage objects in tenancy
64-
6534
### Backup Tablespaces on Source Database
6635

6736
#### Pre-requisites
6837

6938
- Create a Project Directory that will used as staging location on the host running the source database.
7039
- Download [Oracle Database Backup Cloud Module](https://www.oracle.com/database/technologies/oracle-cloud-backup-downloads.html) to the Project Directory. Unzip the downloaded opc_installer.zip in the project directory.
71-
- Download [tts-backup-python.zip](file:////confluence/download/attachments/11465427748/tts-backup-python.zip%3fversion=33&modificationDate=1764740426000&api=v2) to the project directory. Unzip the tts-backup-python.zip in the project directory.
72-
- Provide inputs for backup in the **tts-backup-env.txt** file.
73-
- [tts-backup-python.zip for VodaFone](file:////confluence/download/attachments/11465427748/tts-backup-python.zip%3fversion=33&modificationDate=1764740426000&api=v2)
40+
- Download Transportable Tablespaces Backup Utility files to the project directory.
41+
- Provide inputs for backup in the **tts-backup-env.txt** file.
7442

7543
#### TTS Backup Tool inputs
7644

@@ -90,10 +58,10 @@ Open tts-backup-env.txt file downloaded to the project directory and provide the
9058
***DB_SVC_NAME*** : (REQUIRED INPUT) Database service name, used for connecting to the database.  \
9159
***ORAHOME*** : (REQUIRED INPUT) Database home, \$ORACLE_HOME.\
9260
***DBUSER*** : (REQUIRED INPUT) Username for connecting to the database. User should have sysdba privileges. \
93-
***DBPASSWORD*** : (REQUIRED INPUT) Password for connection to the database. (Provide as CLI argument or Runtime input)\
61+
***DBPASSWORD*** : (RUNTIME INPUT) Password for connection to the database. DO NOT provide in the file. Provide as CLI runtime input when prompted.\
9462
***DB_VERSION*** : (REQUIRED INPUT) DB Version, supported values are 11g, 12c, 19c, 23ai.
9563

96-
##### Object Storage Service (OSS) inputs ( Required if using OSS for transport. Leave them empty if using FSS. )
64+
##### Object Storage Service (OSS) inputs ( Required if using OCI Object Storage for migration. Leave them empty if using FSS. )
9765

9866
***TTS_BACKUP_URL*** : (REQUIRED INPUT) Object storage bucket uri for backup files.      \
9967
***TTS_BUNDLE_URL*** : (REQUIRED INPUT) Object storage bucket uri for transportable tablespace bundle.\
@@ -103,20 +71,22 @@ Open tts-backup-env.txt file downloaded to the project directory and provide the
10371
***OCI_PROXY_HOST :** ***(OPTIONAL INPUT) HTTP proxy server.\
10472
***OCI_PROXY_HOST :** ***(OPTIONAL INPUT)  HTTP proxy server connection port.
10573

106-
##### File Storage Service (FSS) inputs ( Required if using FSS for transport. Leave them empty if using OSS. )
74+
##### File Storage Service (FSS) inputs ( Required if using FSS for migration. Leave them empty if using OSS. )
10775

10876
***TTS_FSS_CONFIG*** : (REQUIRED INPUT) FSS configuration should given in the format FSS:\<FIle System Name\>:\<FQDN of Mount Target\>:\<Export Path\>\
109-
***TTS_FSS_MOUNT_DIR*** : (REQUIRED INPUT) Absolute path where file system was mounted on source database host\
77+
***TTS_FSS_MOUNT_DIR*** : (REQUIRED INPUT) Absolute path where file system was mounted on source database host\
11078
\
111-
Refer to [<https://blogs.oracle.com/datawarehousing/post/attach-file-system-autonomous-database>] for details on how File System should be configured for use by ADB-S.
79+
Refer to [<https://blogs.oracle.com/datawarehousing/post/attach-file-system-autonomous-database>]{.underline} for details on how FIle System should be configured for use by ADB-S.
11280

11381
##### TDE keys inputs
11482

115-
***TDE_WALLET_STORE_PASSWORD*** : (REQUIRED only if any of the tablespaces are TDE Encrypted). Required to export TDE KEYS. (Provide as CLI argument or Runtime input)
83+
***TDE_WALLET_STORE_PASSWORD*** : (RUNTIME INPUT, REQUIRED only if any of the tablespaces are TDE Encrypted). DO NOT provide in the file. Provide as CLI runtime input when prompted.
11684

11785
##### Final backup inputs 
11886

119-
***FINAL_BACKUP*** : (REQUIRED INPUT) Final backup will be done only if FINAL_BACKUP=yes or YES. Accepted values YES, yes, NO, no. Used to indicate incremental operation. Specify YES for non-incremental operation. Specify NO for incremental backups. Last operation should be run with YES for schema to be exported.
87+
***FINAL_BACKUP*** : (REQUIRED INPUT) Accepted values TRUE/true or FALSE/false. Value should be TRUE for a non-incremental migration. For incremental migration,
88+
value should be FALSE for all incremental backups including the first backup. Set the value to TRUE for final backup of an incremental migration. Tablespaces
89+
being transported must be set to READ-ONLY when FINAL_BACKUP is set to TRUE. Tablespace and Schema metadata will be exported only when FINAL_BAKCUP is set to TRUE.
12090

12191
##### Performance inputs     
12292

@@ -125,55 +95,62 @@ Refer to [<https://blogs.oracle.com/datawarehousing/post/attach-file-system-auto
12595

12696
Leave these a blank unless really needed.
12797

128-
### Create or Modify ADB-S database with TTS tag
98+
**Backup Utility Sample Inputs**
99+
100+
Here is an example of [backup utility sample inputs] (https://https://github.com/oracle-devrel/oracle-autonomous-database-samples/tree/main/migration-tools/tts-backup-python/tts_backup_utility_sample_inputs.txt
129101

130-
#### Create ADB-S database to transport tablespaces
102+
Run the TTS Backup Tool from the project directory as below. User will be prompted for database password and optional TDE wallet store password.
131103

132-
Use the below steps to transport tablespaces while creating an ADB-S database.
104+
**Run Backup Utility**
105+
```
106+
$ python3 tts-backup.py 
107+
Enter value for required variable DBPASSWORD: Test
108+
Enter value for optional variable TDE_WALLET_STORE_PASSWORD 
109+
Required only if any of the tablespaces are TDE encrypted (leave empty and press Enter if not applicable):
133110
134-
1. Go to OCI Console → Oracle Database → Autonomous Database
135-
2. Click on ***Create Autonomous Database***
136-
3. Provide all the necessary inputs.
137-
4. Select database version that is equal to or greater than the source database.
138-
5. Specify ***Storage (TB)*** in ***Configure the database*** section to match size of tablespace(s) being transported.
139-
6. Click on ***Show advanced options*** at the bottom of the page. Click on ***Tags*** tab in the section.
140-
7. Select ***Tag namespace*** **as ***None (add a free-form tag)***, ***Tag key*** as ***ADB\$TTS_BUNDLE_URL***,
141-
**Tag value** as metadata bundle url given by the TTS backup tool.
142-
8. Click on ***Add Tag.***
143-
9. Submit ***Create Autonomous Database.***
111+
```
112+
The tool will take backups of the tablespace datafiles and create a metadata bundle. Both backups and the bundle will be uploaded to the provided OCI Object Storage buckets or FSS Mount Targets. Backup Utility will output an URL to OCI Object Storage metadata bundle or FSS Mount Target path for metadata bundle. User should note the given URL/Path as that will be needed as migration input when creating Autonomous AI Database for the migration.
144113

145-
The operation will first create the database and then trigger transport tablespaces job.
114+
### Create Dynamic Group and Policy (for OCI Object Storage backups)
115+
116+
Transportable Tablespaces functionality will download metadata from OCI Object Storage metadata bucket using [OCI Resource Principal](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/resource-principal.html). Create Dynamic Group and Policy to allow access to the metadata bucket using the resource principal. This step is not required if using FSS for backups.
117+
118+
1. Create a Dynamic Group **TTSDynamicGroup** with matching rule:\
119+
ALL {resource.type = \'autonomousdatabase\'[resource.compartment.id](http://resource.compartment.id) = \'your_Compartment_OCID\'}
120+
2. Create a Policy using the dynamic group with Policy Statement:\
121+
Allow dynamic-group **TTSDynamicGroup** to manage buckets in tenancy\
122+
Allow dynamic-group **TTSDynamicGroup** to manage objects in tenancy\
123+
\
124+
Prepend domain name to the dynamic group name if needed as below.\
125+
Allow dynamic-group \<*identity_domain_name\>*/**TTSDynamicGroup** to manage buckets in tenancy\
126+
Allow dynamic-group \<*identity_domain_name\>*/**TTSDynamicGroup** to manage objects in tenancy
146127

147-
#### Modify ADB-S database to transport tablespaces
128+
### Create Autonomous AI Database with Migration inputs
148129

149-
Use the below steps to transport tablespaces to an existing database.
130+
Create an Autonomous AI Database from OCI Console using the steps below. 
150131

151-
1. Go to OCI Console → Oracle Database → Autonomous Database
152-
2. Select and click on the database for transporting tablespaces
153-
3. Verify ***Storage*** in ***Resource allocation*** section. Use ***Manage resource allocation*** tab to increase storage if needed.
154-
4. If this is the first time specifying **ADB\$TTS_BUNDLE_URL** tag on the database:
155-
a. Go to ***More Actions* *→* *Tags** ****menu item on the Autonomous Database Details page.*
156-
b. Select ***Tag namespace*** **as ***None (add a free-form tag)***, ***Tag key*** as ***ADB\$TTS_BUNDLE_URL***, \
157-
**Tag value** as metadata bundle url given by the TTS backup tool.
158-
c. Click on ***Add Tag.***
159-
d. Submit ***Add Tags***.
160-
5. If **ADB\$TTS_BUNDLE_URL** tag was already specified during create or a previous update of the database:
161-
a. Click on the **Tags** tab on the Autonomous Database Details page.
162-
b. Click on **Free-Form tags** tab and edit the **ADB\$TTS_BUNDLE_URL** tag. 
163-
c. Specify the new URL and submit **Save** action.
132+
1. Go to OCI Console → Oracle Database → Autonomous AI Database
133+
2. Click on ***Create Autonomous Database***
134+
3. Provide all the necessary inputs.
135+
4. Select database version that is equal to or greater than the source database.
136+
5. Specify ***Storage (TB)*** in ***Configure the database*** section to match size of tablespace(s) being transported.
137+
6. Expand ***Migration section*** on the page.
138+
7. Specify OCI Object Storage URL or FSS Mount Target path to the metadata bundle as given by Backup Utility.
139+
8. Submit ***Create Autonomous Database.***
164140

165-
The operation will trigger the transport tablespaces job on the database.
141+
The operation will first create the database and then trigger migration operation. For non-incremental operation, migration will perform all the necessary steps and the created database will have the transported tablespaces. In case of incremental operation, this is the first step in the migration process. First backup pieces will be restored to the created database. User has to continue with incremental backup up to the final backup to complete the migration process.
166142

167-
#### To transport tablespaces using incremental database backups
143+
### Incremental Migration
168144

169-
Create a database or update an existing database by specifying **ADB\$TTS_BUNDLE_URL** of level 0 backup.\
170-
For each incremental and final backup, edit the tag using the URL corresponding to that backup as mentioned in **Step 5** above. \
171-
Before taking the final backup, alter all tablespaces being transported as read-only. Specify FINAL_BACKUP as YES in tts-backup-env.txt.\
172-
Datafiles with incremental changes are restored to ADB-S from level 0 to final step. Metadata is imported on final step.
145+
If user is performing an incremental migration operation, repeat **Backup Tablespaces** step at source database for each incremental using the same set of inputs. Backup Utility will output a new OCI Object Storage URL / FSS Mount Target path corresponding to that increment. Use the URL/Path to update the Autonomous AI Database created with the first backup above. Set FINAL_BACKUP=TRUE in the input file before performing final backup.
173146

174-
#### To transport tablespaces using non-incremental database backups
147+
#### Modify Autonomous AI Database with Migration inputs
175148

176-
Non-incremental is a one time operation where datafiles are restored and metadata is imported to ADB-S.\
177-
Alter all tabalespaces being transported as read-only. Specify FINAL_BACKUP as YES in tts-backup-env.txt.\
178-
Create a database or update an existing database by specifying **ADB\$TTS_BUNDLE_URL** of one time backup.
149+
1. Go to OCI Console → Oracle Database → Autonomous AI Database
150+
2. Select and click on the database created with the first backup.
151+
3. Verify ***Storage*** in ***Resource allocation*** section. Use ***Manage resource allocation*** tab to increase storage if needed.
152+
4. Expand ***Migration section*** on the page.
153+
5. Specify OCI Object Storage URL or FSS Mount Target path to the metadata bundle as given by Backup Utility for the increment or final backup.
154+
6. Click on **Save**.
179155

156+
The operation will trigger the transport tablespaces job on the database.
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
################################################################################
2+
### Project and Tablespace inputs ###
3+
################################################################################
4+
PROJECT_NAME=tts_project
5+
DATABASE_NAME=orclpdb1
6+
TABLESPACES=emptbs,salestbs
7+
SCHEMAS=user1,user2,user3
8+
9+
10+
################################################################################
11+
### Database connection inputs ###
12+
################################################################################
13+
HOSTNAME=mydbhost1
14+
LSNR_PORT=1521
15+
DB_SVC_NAME=orclpdb1.company.com
16+
ORAHOME=/opt/oracle/product/19c/dbhome_1
17+
DBUSER=mydbuser
18+
DB_VERSION=19c
19+
20+
21+
################################################################################
22+
### Object Storage Service (OSS) inputs ###
23+
################################################################################
24+
# The following inputs are required only in case of OSS based transport ###
25+
# Please leave them empty if using FSS. ###
26+
################################################################################
27+
TTS_BACKUP_URL=https://objectstorage.us-ashburn-1.oraclecloud.com/n/mynamespace1/b/tts-backup
28+
TTS_BUNDLE_URL=https://objectstorage.us-ashburn-1.oraclecloud.com/n/mynamespace1/b/tts-metadata
29+
OCI_INSTALLER_PATH=/home/oracle/opc_installer/oci_installer/oci_install.jar
30+
CONFIG_FILE=/home/oracle/OCI_CONFIG/config
31+
COMPARTMENT_OCID=ocid1.compartment.oc1..xxxxx
32+
OCI_PROXY_HOST=proxy.company.com
33+
OCI_PROXY_PORT=80
34+
35+
36+
################################################################################
37+
### File Storage Service (FSS) inputs ###
38+
################################################################################
39+
# The following inputs are required only in case of FSS based transport ###
40+
# Please leave them empty if using Object storage. ###
41+
################################################################################
42+
TTS_FSS_CONFIG=FSS:myFSS:myfss.vcn.com:/myFSS
43+
TTS_FSS_MOUNT_DIR=/u01/app/oracle/myFSS
44+
45+
################################################################################
46+
### Final backup inputs ###
47+
################################################################################
48+
FINAL_BACKUP=TRUE/FALSE
49+
50+
51+
################################################################################
52+
### Performance inputs ###
53+
################################################################################
54+
PARALLELISM=
55+
CPU_COUNT=

0 commit comments

Comments
 (0)