Backup and restore is a core component of the high availability feature of OceanBase AI Database. It is mainly used to ensure data security, including preventing storage media damage and user errors. If data is lost due to storage media damage or user errors, you can restore the data by using the restore feature.
Notice
The physical backup feature is supported starting from V4.6.2 BP1.
Overview
The backup and restore module of OceanBase AI Database provides features such as backup, restore, and cleanup.
Backup
OceanBase AI Database supports tenant-level physical backup. A physical backup consists of data backup and log archiving. Therefore, the physical backup feature is a combination of data backup and log archiving. Here, a tenant refers to a user tenant. Physical backup is not supported for the sys tenant or the Meta tenant.
Data backup refers to the feature of backing up data. It supports full backup, which backs up all macroblocks.
Notice
Before you perform a physical backup, you must enable the log archiving mode.
The data backed up by data backup includes the following:
Information about the tenant, including the tenant name, cluster name, time zone (timezone), replica distribution (locality), and compatibility mode (MySQL) of the tenant.
All user table data.
Note
Data backup backs up system variables and tenant-level configuration items, but does not back up cluster-level configuration items or private system table data.
Log archiving refers to the automatic archiving of log data. OBServer nodes periodically archive log data to the specified backup path. This process is fully automatic and does not require external triggering. After the log archiving service is enabled, the data dictionary is also archived.
Restore
The overall architecture of physical restore is as follows:
Physical restore supports tenant-level restore: the process of rebuilding a new tenant based on the existing data backup. Tenant-level restore ensures global consistency across tables and partitions.
Tenant-level restore supports full restore.
Full restore refers to restoring macroblock data and incremental logs. After all data is restored from the backup media to the local environment, the restored tenant can provide services. The full restore process includes the restore and recover processes for tenant system tables and user tables. Restore refers to restoring the baseline data required for the restore to the OBServer node of the target tenant. Recover refers to restoring the logs corresponding to the baseline to the corresponding OBServer node.
Select the restore time point for your physical restore.
Complete restore: No restore timestamp is specified.
Incomplete restore with a specified SCN or timestamp: SCN is the precise version number of OceanBase AI Database. The timestamp is precise to microseconds, and precision beyond microseconds is lost.
Cleanup
OceanBase AI Database provides automatic and manual cleanup capabilities for backup data, so that you can flexibly handle different business scenarios.
Automatic cleanup: An automatic cleanup mechanism based on the recovery window (recovery_window). After you set the expiration time for backup data, the system periodically triggers cleanup tasks to remove expired backup sets and data archive pieces.
Manual cleanup: Manual cleanup is divided into two types. The first type is manual cleanup of specified backup sets or log archive pieces. The second type is manual cleanup of all backup sets or data archive pieces in the entire path (data backup path or log archive path).
Backup media requirements
OceanBase AI Database of the current version supports backup media such as Azure Blob, AWS S3, and object storage services that are compatible with the S3 protocol (for example, Alibaba Cloud OSS, Huawei OBS, Google GCS, and Tencent Cloud COS). Some of these backup media require meeting certain basic requirements before they can be used.
SDK version requirements
In the current version, obdal is used by default to access object storage. obdal is a unified framework for object storage developed based on OpenDAL. For more information about OpenDAL, see OpenDAL.
In addition, if you want to use the SDK of an object storage service, the version of the SDK must be 1.11.156.
Interface requirements
Object storage services compatible with the S3 protocol (for example, Alibaba Cloud OSS, Huawei OBS, Google GCS, and Tencent Cloud COS):
The following table describes the S3 APIs that must be supported.
Interface nameDescriptionPutObject Uploads a single object. DeleteObject Deletes a single object. DeleteObjects Deletes objects in batches. GetObject Downloads a single object. ListObjects Lists all objects under the path. HeadObject Retrieves the metadata of an object. PutObjectTagging (Optional) Sets the tags of an object. GetObjectTagging (Optional) Retrieves the tags of an object. CreateMultipartUpload Initializes a multipart upload. UploadPart Uploads a single part. CompleteMultipartUpload Merges the uploaded parts into one object. AbortMultipartUpload Aborts a multipart upload and deletes the uploaded parts. ListMultipartUploads Lists the uploaded parts. ListParts Lists the uploaded parts in an upload task. The object access URL must support the virtual-hosted-style access method. For more information about virtual-hosted-style requests, see AWS S3 documentation.
Before you select a backup medium, you can run the test_io_device command in the ob_admin tool to verify whether the I/O interfaces and I/O permissions provided by the backup medium meet the requirements for backup and restore. You can also run the io_adapter_benchmark command in the ob_admin tool to view the read and write performance between an OBServer node and the backup medium, which can be used as a reference for backup performance.
Directory structure
Data backup directory
The directories created by the data backup feature at the backup destination and the types of files stored in each directory are as follows:
data_backup_dest
├── format.obbak // Formatted information of the backup path
├── check_file
│ └── 1002_connect_file_20230111T193020.obbak // Connectivity check file
├── backup_sets // A summary directory of the data backup list, which records the list of all data backup sets
│ ├── backup_set_1_full_end_success_20230111T193420.obbak // Placeholder for the end of a full backup
│ ├── backup_set_1_full_start.obbak // Placeholder for the start of a full backup
└── backup_set_1_full // A full backup set. A name ending with full indicates a full backup
├── file_list.0.obbak
├── backup_set_1_full_20230111T193330_20230111T193420.obbak // Placeholder that shows the start and end times of the full backup
├── single_backup_set_info.obbak // Metadata of the current backup set
├── tenant_backup_set_infos.obbak // Information about the full backup sets of the current tenant
├── infos // Metadata directory of the backup set
├── logstream_1 // Log stream 1
├── logstream_1001 // Log stream 1001
├── ls // Log stream directory
└── tablet // Tablet directory
In the data backup directory, the top-level directory contains the following three types of data:
format.obbak: used to record the metadata of the backup path.check_file: used for connectivity check of the user data backup directory.backup_sets: a summary directory of the list of data backup sets, which records the list of all data backup sets.backup_set_1_full: the directory indicates a data backup set.backup_set_1indicates that thebackup_set_idof the backup set is1. If the directory name ends withfull, it indicates a full backup. A backup set is generated for each data backup. After the data backup is completed, the backup set will no longer be modified.A data backup set mainly contains the following data:
file_list.0.obbak: this file records the files and directories contained in the current directory.backup_set_1_full_20230111T193330_20230111T193420.obbak: this file shows the ID, start time, and end time of the current backup set. This file is only for information display.single_backup_set_info.obbak: this file records the metadata of the current backup set, including the backup point and the dependent logs.tenant_backup_set_infos.obbak: this file records the metadata of all existing backup sets of the current tenant.infos: this directory records the metadata of the data backup set.logstream_1: this directory records all data of log stream 1. Log stream 1 is the system log stream of the OceanBase AI Database tenant.logstream_1001: this directory records all data of log stream 1001. Log streams with numbers greater than 1000 are the user log streams of the OceanBase AI Database tenant.ls: the log stream directory. Its structure is consistent with the directory structure of shared data on object storage.tablet: the tablet directory. Its structure is consistent with the directory structure of shared data on object storage.
Cluster-level configuration item backup directory
Each time a backup of cluster-level configuration items is initiated, the system generates a backup file of the cluster-level configuration items in the specified directory. The directory structure is as follows:
cluster_parameters_backup_dest
├── cluster_parameter.20240710T103610.obbak # Information of non-default cluster-level configuration items. File naming format: `cluster_parameter.[timestamp]`
└── cluster_parameter.20241018T140609.obbak
Log archive directory
For backup media such as Azure Blob, the directories created by the log archiving feature at the archive destination and the types of files stored in each directory are as follows:
log_archive_dest
├── check_file
│ └── 1002_connect_file_20230111T193049.obbak // Connectivity check file
├── format.obbak // Formatted information of the backup path
├── rounds // Placeholder directory for Rounds
│ └── round_d1002r1_start.obarc // Placeholder for Round start
├── pieces // Placeholder directory for Pieces
│ ├── piece_d1002r1p1_start_20230111T193049.obarc // Placeholder for Piece start, named as piece_DESTID_ROUNDID_PIECEID_start_DATE
│ └── piece_d1002r1p1_end_20230111T193249.obarc // Placeholder for Piece end, named as piece_DESTID_ROUNDID_PIECEID_end_DATE
└── piece_d1002r1p1 // Piece directory, named as piece_DESTID_ROUNDID_PIECEID
├── file_list.0.obarc
├── piece_d1002r1p1_20230111T193049_20230111T193249.obarc // Records the continuous interval of the Piece
├── checkpoint
├── single_piece_info.obarc // Records the metadata of the current Piece
├── tenant_archive_piece_infos.obarc // Records the metadata of all frozen Pieces before the current Piece
├── file_info.obarc // List of all log stream files
├── logstream_1 // Log stream 1
└── logstream_1001 // Log stream 1001
In the above log archive directory, the top-level directory contains the following three types of data:
format.obbak: records the metadata of the archive path, including information such as the tenant using the path.check_file: used for connectivity check of the user log archive directory.rounds: a summary list of all Rounds in the log archive, recording the list of all Rounds.pieces: a summary list of all Pieces in the log archive, recording the list of all Pieces.piece_d1002r1p1: the directory for a Piece in the log archive, named aspiece_DESTID_ROUNDID_PIECEID. Here,DESTIDrefers to the id corresponding tolog_archive_dest;ROUNDIDrefers to the id of the log archive Round, which is a monotonically increasing integer; andPIECEIDrefers to thepiece_idof the log archive Piece, which is also a monotonically increasing integer.A log archive Piece directory contains the following data:
file_list.0.obarc: this file records the files and directories contained in the current directory.piece_d1002r1p1_20230111T193049_20230111T193249.obarc: this file displays the id, start, and end times of the current Piece and is only for informational purposes.checkpoint: this directory is used by the active Piece to record the archive point. The ObArchiveScheduler module periodically updates the archive point information in this directory.single_piece_info.obarc: this file records the metadata of the current Piece.tenant_archive_piece_infos.obarc: this file records the metadata of all frozen Pieces in the current tenant.file_info.obarc: this file records the list of log stream files within the Piece.logstream_1: this directory records the log files of log stream 1, which is the system log stream of the OceanBase AI Database tenant.logstream_1001: this directory records the log files of log stream 1001. Log streams with numbers greater than 1000 are the user log streams of the OceanBase AI Database tenant.