OceanBase AI Database adopts a shared storage architecture based on storage and compute separation. Incremental data, baseline data, and log files are all stored on object storage.
Object storage serves as the data foundation for OceanBase AI Database, providing shared persistent storage capabilities for the compute layer and the LogService layer. It is responsible for the long-term preservation and unified access of full data sets, and is the core foundation for achieving low-cost, highly reliable data storage. In this architecture, compute nodes (OBServer) share the full data set on object storage, caching only hot data locally to ensure access performance; the log service (LogService) achieves low-latency writing of the latest logs through local high-performance storage, and asynchronously archives confirmed logs to object storage for massive persistent storage.
Before you create a cluster, you must configure the connection information of object storage by using the External Integration feature of OceanBase Cloud Platform (OCP), including the bucket path, endpoint address, and AccessKey credentials. For more information, see Create an object storage configuration.
Components that use object storage
In the shared storage architecture, object storage serves multiple components within the cluster, with each component accessing object storage using independent credential management methods:
Component |
Purpose |
Management method |
|---|---|---|
| OBServer (compute node) | Persists baseline and incremental data | Specify SHARED_STORAGE_INFO by using the bootstrap command during cluster initialization. The data storage credentials are passed through the ALTER SYSTEM CHANGE SHARED_STORAGE_DEST statement, and the log storage credentials are updated by modifying the logservice_access_point parameter. |
| LogService (log service node) | Persists log data | Rotate access credentials (AK/SK) by using the ls_ctrl rotate-ak command-line tool. |
Concepts
Object storage: a storage architecture for massive unstructured data that stores data as independent objects containing metadata and a unique identifier in a flat namespace, and accesses the objects through APIs. OceanBase AI Database currently supports only AWS S3-compatible object storage and Azure Blob Storage.
Bucket: a container in object storage that is used to store objects (files). Each bucket has a unique domain name and access address.
Endpoint: the access address of the object storage service. Different regions have different endpoints.
AccessKey: the credentials for accessing object storage, consisting of an AccessKey ID and an AccessKey Secret, used for authentication.
Management
The configuration of object storage is centrally managed by using the External Integration feature of OCP. It provides:
- Unified management: Centrally manage the object storage configurations used by clusters, avoiding configuration inconsistencies caused by decentralized management.
- Lifecycle management: Manage the complete lifecycle of object storage configurations, including adding, updating, and deleting configurations. Deleting a configuration only removes the configuration record from OCP and does not delete the actual data in the object storage service.
- Credential security: Securely store AccessKey credentials, which are automatically injected during cluster creation, eliminating the need for manual entry.
Considerations
Independent buckets: It is recommended that you configure separate object storage buckets for OBServer and LogService to ensure data isolation and security.
Least privilege principle: Configure the minimum necessary permissions for the object storage AccessKey, following the principle of least privilege.
Regular rotation: Regularly rotate AccessKeys to reduce the risk of credential leakage.
Supported storage types
Currently, OceanBase AI Database can connect to AWS S3-compatible object storage (such as Alibaba Cloud OSS and Huawei Cloud OBS) and Azure Blob Storage. The corresponding URL prefixes are s3:// and azblob://. Only these two protocols are supported.
Supported interfaces and permission requirements
OceanBase AI Database accesses object storage through the S3-compatible protocol and supports the following APIs:
API name |
Description |
|---|---|
PutObject |
Upload a single object |
DeleteObject |
Delete a single object |
DeleteObjects |
Delete multiple objects in batches |
GetObject |
Download a single object |
ListObjects |
List all objects in the path |
HeadObject |
Retrieve metadata of a specific object |
CreateMultipartUpload |
Initialize a multipart upload |
UploadPart |
Upload a part |
CompleteMultipartUpload |
Complete a multipart upload |
AbortMultipartUpload |
Abort a multipart upload and delete the uploaded parts |
ListMultipartUploads |
List multipart uploads |
ListParts |
List the parts in a multipart upload |
PutObjectTagging |
Set tags for a single object |
GetObjectTagging |
Get tags of a single object |
The minimum permissions required for different components to access object storage are as follows:
OBServer (compute node): requires permissions for all the preceding APIs to persist baseline and incremental data.
LogService (log service node): requires permissions for the following APIs to upload, read, and clean up log data:
APIDescriptionPutObjectUpload a log block GetObjectRead a log block HeadObjectCheck whether an object exists ListObjectsList objects (for garbage collection scans, etc.) DeleteObjectDelete a single object (for garbage collection cleanup) DeleteObjectsDelete multiple objects in batches (S3). On Azure Blob Storage, this is equivalent to executing multiple DeleteObjectcalls.
Object storage status
The lifecycle status of object storage is defined by the STATE field in the DBA_OB_ZONE_STORAGE view, including:
Status |
Description |
|---|---|
ADDING |
Adding object storage |
ADDED |
Added successfully. The object storage is available. |
DROPPING |
Deleting object storage |
CHANGING |
Rotating AK/SK |
CHANGED |
AK/SK rotated |