OceanBase AI Database provides load balancing capabilities such as horizontal scaling and dynamic data balancing.
Horizontal scaling refers to the ability to scale service capacity by adjusting the number of service nodes. For example, scaling out from a single service node to two service nodes increases service capacity. Horizontal scaling also requires data redistribution. For instance, when scaling out from a single service node to two service nodes, data must be evenly distributed across the two nodes. Conversely, when scaling in from two service nodes to one, data must be redistributed onto that single node.
Dynamic data balancing refers to the ability to dynamically balance the load across service nodes by adjusting data distribution while keeping the number of service nodes constant. For example, as tables and partitions are dynamically created and deleted, the number of partitions on different service nodes can vary significantly, leading to uneven load distribution. With dynamic partition balancing, partitions can be evenly distributed across service nodes, achieving load balancing.
Horizontal scaling
In OceanBase AI Database, a tenant's storage capacity and read/write service capability are primarily influenced by the following two factors:
Unit Number, which is the number of service units provided per zone.
By increasing or decreasing the Unit Number, you can add or remove service nodes, thereby implementing horizontal scaling for read/write services and storage capacity.
For more information about how to scale a tenant by adjusting the Unit Number, see Scale tenants by adjusting the unit number.
Primary Zone, which is the list of zones providing read/write services.
By increasing or decreasing the number of primary zones with the highest priority, you can add or remove zones providing read/write services, thereby implementing horizontal scaling for read/write services across zones.
For more information about how to scale a tenant by adjusting the Primary Zone, see Scale tenants by adjusting the primary zone.
By dynamically adjusting the Unit Number and Primary Zone, you can implement horizontal scaling for a tenant's read/write service capability within and between zones. The load balancing feature will adaptively adjust log streams and partition distribution based on the configured service capability.
Partition balancing
Partition balancing refers to the process of dynamically adjusting partition distribution to achieve equilibrium in the number of partitions, partition weights, table group weights, and storage space across service nodes, despite the dynamic changes in tables and partitions.
OceanBase AI Database supports multiple table types, including non-partitioned tables, partitioned tables, and subpartitioned tables. Different table types have different balancing strategies. To facilitate the description of balancing effects, OceanBase AI Database groups partitions into balancing groups. Within each balancing group, equilibrium in the number of partitions and storage space must be achieved. There is no direct relationship between balancing groups, and the distribution between them is adaptively adjusted internally. By default, the partition balancing strategy in OceanBase AI Database is as follows:
Partitioned tables: Each partitioned table is an independent balancing group. All partitions of the table are scattered across various service nodes.
Subpartitioned tables: All subpartitions under each partition form an independent balancing group. All subpartitions under each partition are scattered across various service nodes.
Non-partitioned tables: All non-partitioned tables are considered together and form one and only one balancing group. All non-partitioned tables are scattered across various service nodes.
To more flexibly describe the clustering and scattering relationships between data of different tables, OceanBase AI Database introduces the concept of Table Group.
A table group is a logical concept representing a collection of tables. Tables within a table group are physically stored adjacent to each other. Tables with related associations often have the same partitioning rules. By clustering partitions with the same rules together, Partition Wise Join can be achieved, greatly optimizing read and write performance.
OceanBase AI Database uses the SHARDING attribute of a table group to control the clustering and scattering relationships of table data within the group. The SHARDING attribute has three possible values: NONE, PARTITION, and ADAPTIVE.
You can choose an appropriate SHARDING attribute value based on your usage scenarios.
Scenario 1: All tables within a table group are clustered together
If you want to aggregate tables of any type on one server to meet single-machine access requirements, you can use a table group with SHARDING = NONE to group tables of any type together.
The meaning of SHARDING = NONE for a table group is as follows:
- It supports adding tables of any partition type, including non-partitioned tables, partitioned tables, and subpartitioned tables.
- It aggregates all partitions of all tables within the table group together, ensuring that scheduling occurs on one server.
Scenario 2: Horizontal distribution of data within tables in a table group
When a single server cannot handle the data of a single business, and you want to distribute the data across multiple servers for horizontal scaling, you can use a table group with SHARDING = PARTITION or SHARDING = ADAPTIVE to meet this requirement.
The meaning of SHARDING = PARTITION for a table group is as follows:
It supports adding partitioned tables and subpartitioned tables.
- Partition requirement: The partitioning method must be the same for all partitioned tables; for subpartitioned tables, the system only verifies the partitioning method of the parent partition. Therefore, partitioned tables and subpartitioned tables can coexist, as long as they have the same partitioning method.
- Partition alignment rule: Partitions with the same parent partition value are aggregated together, including all subpartitions under the corresponding parent partition of both the partitioned table and the subpartitioned table.
It distributes all tables within the table group by parent partition. For subpartitioned tables, all subpartitions under each parent partition are aggregated together.
The meaning of SHARDING = ADAPTIVE for a table group is as follows:
It supports adding either partitioned tables or subpartitioned tables.
Partition requirement: All tables must be either all partitioned tables or all subpartitioned tables. If they are partitioned tables, they must have the same partitioning method; if they are subpartitioned tables, they must have the same partitioning and subpartitioning methods.
Partition alignment rule:
- Partitioned tables: Partitions with the same parent partition value are aggregated together.
- Subpartitioned tables: Partitions with the same parent partition value and the same subpartition value are aggregated together.
Adaptive distribution method. If all tables in the table group are partitioned tables, they are distributed by parent partition. If all tables in the table group are subpartitioned tables, they are distributed by subpartition under each parent partition.
For more detailed information and operations on table groups, see About table groups.
Balancing based on table group weights
OceanBase AI Database achieves aggregation of user tables by binding a database to a Sharding = 'NONE' table group. It currently supports automatic aggregation of user tables within a single database and aggregation of user tables across multiple databases.
By setting weights for a Sharding = 'NONE' table group, you can achieve balanced table group weights, which helps distribute user tables under automatically aggregated databases according to their weights. Table group weight balancing is a process within the Partition Balancing (PARTITION_BALANCE) task. User tenants can trigger it manually or wait for a scheduled partition balancing task to initiate it.
Balancing based on partition weights
To further address issues such as partition hotspots within partitioned tables and between tables, OceanBase AI Database provides a partition balancing strategy based on weights. Partition weight refers to the relative proportion of resources such as CPU, memory, and disk occupied by different partitions. Partition weight balancing is a process within the Partition Balancing (PARTITION_BALANCE) task. User tenants can trigger it manually or wait for a scheduled partition balancing task to initiate it.
The partition weight balancing algorithm minimizes the variance of the sum of partition weights across user log streams through partition movement and partition exchange.
Only tables with assigned partition weights participate in partition weight balancing. Partitions without assigned weights are still balanced based on the number of partitions.
Data load balancing-related parameters
-
The tenant-level parameter
enable_rebalancecontrols whether to perform load balancing between tenants in the sys tenant and within a tenant in the user tenant. The default value istrue. -
The tenant-level parameter
enable_transfercontrols whether to perform transfers within a tenant. The default value istrue. Specifically:When the value of the
enable_rebalanceparameter isfalse, the system does not perform automatic load balancing, regardless of the value of theenable_transferparameter.When the value of the
enable_rebalanceparameter istrueand the value of theenable_transferparameter is alsotrue, the system automatically adjusts the number of log streams within a tenant during tenant scaling. This is done through operations such as log stream splitting, merging, and transfers, to achieve leader balance and partition balance within the tenant.When the value of the
enable_rebalanceparameter istrueand the value of theenable_transferparameter isfalse, the system does not perform transfers during tenant scaling, nor does it change the number of log streams. Instead, it strives to maintain log stream balance based on the existing log streams.
partition_balance_schedule_interval
The tenant-level parameter
partition_balance_schedule_intervalcontrols the time interval for generating partition load balancing tasks. Whenenable_rebalanceis set totrue, the system automatically triggers partition load balancing tasks at intervals determined bypartition_balance_schedule_interval. The default value is2h, with a range of [0s, +∞). A value of0sdisables partition balancing.Notice
We recommend that you do not use this parameter to control partition balancing. You can manually or periodically trigger partition balancing by calling the
DBMS_BALANCE.TRIGGER_PARTITION_BALANCEsubprogram.-
The tenant-level parameter
enable_database_sharding_nonecontrols whether to enable automatic aggregation for user tables when creating a database. The default value isFalse, which means automatic aggregation for user tables is disabled by default when creating a database.