Tenant scaling, essentially, involves increasing or decreasing a tenant's service capabilities, including computing power and storage capacity. This can be achieved by upgrading or downgrading the service capabilities of a single node or by adding or removing compute nodes. This topic describes how to modify the primary zone to horizontally add or remove compute nodes, thereby increasing or decreasing the total service capability of a tenant and achieving tenant scaling.
Prerequisites
Before you perform a scale-in or scale-out operation on a tenant, you must perform the following steps:
Enable load balancing for the tenant.
The load balancing strategy within a tenant is jointly controlled by the tenant-level parameters
enable_rebalanceandenable_transfer.The
enable_rebalanceparameter controls whether to enable load balancing between tenants in thesystenant and whether to enable intra-tenant load balancing in user tenants. When you perform a scale-in or scale-out operation on a tenant, you must enable intra-tenant load balancing. The default value of this parameter istrue, indicating that load balancing is enabled. The modification takes effect immediately without restarting the OBServer node.The
enable_transferparameter controls whether to enable the transfer feature for a tenant. The default value istrue. The modification takes effect immediately without restarting the OBServer node. The semantics of theenable_transferparameter depends on the value of theenable_rebalanceparameter:If the value of
enable_rebalanceisfalse, automatic load balancing is not performed regardless of the value ofenable_transfer.If the value of
enable_rebalanceistrueand the value ofenable_transferistrue, the system automatically adjusts the partition distribution during tenant scaling to achieve load balancing.If the value of
enable_rebalanceistrueand the value ofenable_transferisfalse, the system cannot initiate transfer operations during tenant scaling. Load balancing can only be achieved to a limited extent through log stream migration.
You can enable load balancing for a tenant in the following ways:
Enable load balancing for a specified tenant from the
systenantEnable intra-tenant load balancing and the transfer feature for a specified tenant from the
systenantALTER SYSTEM SET enable_rebalance = true TENANT = 'tenant_name';ALTER SYSTEM SET enable_transfer = true TENANT = 'tenant_name';These statements only set the values of the
enable_rebalanceandenable_transferparameters totruefor the specified user tenant.Enable intra-tenant load balancing and the transfer feature for all user tenants (excluding the
systenant and the Meta tenant) from thesystenantALTER SYSTEM SET enable_rebalance = true TENANT = all_user;ALTER SYSTEM SET enable_transfer = true TENANT = all_user;or
ALTER SYSTEM SET enable_rebalance = true TENANT = all;ALTER SYSTEM SET enable_transfer = true TENANT = all;These statements set the values of the
enable_rebalanceandenable_transferparameters totruefor all user tenants.Note
TENANT = all_userandTENANT = allhave the same semantics. If you want the modification to take effect for all user tenants, we recommend that you useTENANT = all_user.TENANT = allwill be deprecated in the future.
Enable load balancing for the current tenant from a user tenant
The statement to enable intra-tenant load balancing and the transfer feature for the current tenant is as follows:
ALTER SYSTEM SET enable_rebalance = true;ALTER SYSTEM SET enable_transfer = true;
For more information about the
enable_rebalanceandenable_transferparameters, see enable_rebalance and enable_transfer.Plan resources for the tenant before scaling to better achieve the expected result.
For more information about resource planning for tenant scaling, see Resource planning for tenant scaling.
Add a primary zone
Log in to the
systenant of the cluster as therootuser.obclient -h172.30.xxx.xxx -P2883 -uroot@sys#obdemo -pxxxx -AAccess the database named
oceanbase.use oceanbase;Query the basic information of the
mysql001tenant, such as theTENANT_IDandPRIMARY_ZONEattributes.obclient> SELECT TENANT_ID, TENANT_NAME, PRIMARY_ZONE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_NAME = 'mysql001';The query result is as follows:
+-----------+-------------+--------------+ | TENANT_ID | TENANT_NAME | PRIMARY_ZONE | +-----------+-------------+--------------+ | 1004 | mysql001 | zone1;zone2 | +-----------+-------------+--------------+ 1 row in setAs shown in the query result, the
TENANT_IDof tenantmysql001is 1004, and the first priority zone iszone1.Query the distribution of the current log stream leader replica locations for tenant
mysql001.obclient> SELECT TENANT_ID, LS_ID, SVR_IP, SVR_PORT, ZONE, ROLE, REPLICA_TYPE FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE TENANT_ID = 1004 AND LS_ID != 1 AND ROLE = 'LEADER';The query result is as follows:
+-----------+-------+---------------+----------+--------+--------+--------------+ | TENANT_ID | LS_ID | SVR_IP | SVR_PORT | ZONE | ROLE | REPLICA_TYPE | +-----------+-------+---------------+----------+--------+--------+--------------+ | 1004 | 1001 | 172.xx.xxx.xx | 2882 | zone1 | LEADER | FULL | | 1004 | 1012 | 172.xx.xxx.xx | 2882 | zone1 | LEADER | FULL | +-----------+-------+---------------+----------+--------+--------+--------------+ 2 rows in setAs shown in the query result, tenant
mysql001has 2 leader replicas providing services, which are evenly distributed across two OBServer nodes in zone1.Modify the
PRIMARY_ZONEattribute of themysql001tenant to change the number of first-priority zones from 1 to 2.obclient> ALTER TENANT mysql001 PRIMARY_ZONE='zone1,zone2';View the execution status of the job for adding a primary zone.
obclient> SELECT * FROM oceanbase.DBA_OB_TENANT_JOBS WHERE JOB_TYPE='ALTER_TENANT_PRIMARY_ZONE' AND TENANT_ID = 1004;The query result is as follows:
+--------+---------------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+--------------------------------------------------+----------------------------------------+----------------+-------------+ | JOB_ID | JOB_TYPE | JOB_STATUS | RESULT_CODE | PROGRESS | START_TIME | MODIFY_TIME | TENANT_ID | SQL_TEXT | EXTRA_INFO | RS_SVR_IP | RS_SVR_PORT | +--------+---------------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+--------------------------------------------------+----------------------------------------+----------------+-------------+ | 4 | ALTER_TENANT_PRIMARY_ZONE | SUCCESS | 0 | 100 | 2024-12-18 17:26:00.069089 | 2024-12-18 17:26:20.919021 | 1004 | ALTER TENANT mysql001 PRIMARY_ZONE='zone1,zone2' | FROM: 'zone1;zone2', TO: 'zone1,zone2' | 172.xx.xxx.xx | 2882 | +--------+---------------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+--------------------------------------------------+----------------------------------------+----------------+-------------+ 1 row in setIn the query result, find the corresponding job record based on the following fields:
START_TIME: the time when the job was initiated.SQL_TEXT: the SQL statement corresponding to the job.EXTRA_INFO: the primary zone information before and after the change.
If the value of
JOB_STATUSisSUCCESSin the corresponding job record, the job for adding a primary zone is successfully executed.For more information about the fields in the
DBA_OB_TENANT_JOBSview, see DBA_OB_TENANT_JOBS.Query the basic information of the
mysql001tenant. The zones with the highest priority change fromzone1tozone1,zone2, indicating that the modification of thePRIMARY_ZONEattribute is successful.obclient> SELECT PRIMARY_ZONE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_ID = 1004;The query result is as follows:
+--------------+ | PRIMARY_ZONE | +--------------+ | zone1,zone2 | +--------------+ 1 row in setQuery the distribution of the log stream leader replica locations of the
mysql001tenant again.obclient> SELECT TENANT_ID, LS_ID, SVR_IP, SVR_PORT, ZONE, ROLE, REPLICA_TYPE FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE TENANT_ID = 1004 AND LS_ID != 1 AND ROLE = 'LEADER';The query result is as follows:
+-----------+-------+---------------+----------+-------+--------+--------------+ | TENANT_ID | LS_ID | SVR_IP | SVR_PORT | ZONE | ROLE | REPLICA_TYPE | +-----------+-------+---------------+----------+-------+--------+--------------+ | 1004 | 1001 | 172.xx.xxx.xx | 2882 | zone1 | LEADER | FULL | | 1004 | 1012 | 172.xx.xxx.xx | 2882 | zone1 | LEADER | FULL | | 1004 | 1013 | 172.xx.xxx.xx | 2882 | zone2 | LEADER | FULL | | 1004 | 1014 | 172.xx.xxx.xx | 2882 | zone2 | LEADER | FULL | +-----------+-------+---------------+----------+-------+--------+--------------+ 4 rows in setAs shown in the query result, tenant
mysql001has 2 leader replicas in each of zones 1 and 2, totaling 4 leader replicas providing services.
The preceding example shows that before the change, tenant mysql001 has 1 zone with the highest PRIMARY_ZONE priority and 2 leader replicas providing services. After the change, the number of zones with the highest PRIMARY_ZONE priority for tenant mysql001 increases from 1 to 2, and the number of leader replicas providing services increases from 2 to 4. The compute nodes are added, enabling the tenant to scale out.
Reduce primary zones
Log in to the
systenant of the cluster as therootuser.obclient -h172.30.xxx.xxx -P2883 -uroot@sys#obdemo -pxxxx -AAccess the database named
oceanbase.use oceanbase;Query the basic information of the
mysql001tenant, such as theTENANT_IDandPRIMARY_ZONEattributes.obclient> SELECT TENANT_ID, TENANT_NAME, PRIMARY_ZONE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_NAME = 'mysql001';The query result is as follows:
+-----------+-------------+--------------+ | TENANT_ID | TENANT_NAME | PRIMARY_ZONE | +-----------+-------------+--------------+ | 1004 | mysql001 | zone1,zone2 | +-----------+-------------+--------------+ 1 row in setAs shown in the query result, the
TENANT_IDof themysql001tenant is 1004, and the first priority zones arezone1andzone2.Query the distribution of leader replicas of the current log stream for tenant
mysql001.obclient> SELECT TENANT_ID, LS_ID, SVR_IP, SVR_PORT, ZONE, ROLE, REPLICA_TYPE FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE TENANT_ID = 1004 AND LS_ID != 1 AND ROLE = 'LEADER';The query result is as follows:
+-----------+-------+---------------+----------+-------+--------+--------------+ | TENANT_ID | LS_ID | SVR_IP | SVR_PORT | ZONE | ROLE | REPLICA_TYPE | +-----------+-------+---------------+----------+-------+--------+--------------+ | 1004 | 1001 | 172.xx.xxx.xx | 2882 | zone1 | LEADER | FULL | | 1004 | 1012 | 172.xx.xxx.xx | 2882 | zone1 | LEADER | FULL | | 1004 | 1013 | 172.xx.xxx.xx | 2882 | zone2 | LEADER | FULL | | 1004 | 1014 | 172.xx.xxx.xx | 2882 | zone2 | LEADER | FULL | +-----------+-------+---------------+----------+-------+--------+--------------+ 4 rows in setAs shown in the query result, tenant
mysql001has 4 leader replicas providing services, which are evenly distributed across two OBServer nodes in each of zones 1 and 2.Modify the
PRIMARY_ZONEattribute of themysql001tenant to reduce the number of first-priority zones from 2 to 1.obclient> ALTER TENANT mysql001 PRIMARY_ZONE='zone1;zone2';View the execution status of the job for reducing primary zones.
obclient> SELECT * FROM oceanbase.DBA_OB_TENANT_JOBS WHERE JOB_TYPE='ALTER_TENANT_PRIMARY_ZONE' AND TENANT_ID = 1004;The query result is as follows:
+--------+---------------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+--------------------------------------------------+------------------------------------------+----------------+-------------+ | JOB_ID | JOB_TYPE | JOB_STATUS | RESULT_CODE | PROGRESS | START_TIME | MODIFY_TIME | TENANT_ID | SQL_TEXT | EXTRA_INFO | RS_SVR_IP | RS_SVR_PORT | +--------+---------------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+--------------------------------------------------+------------------------------------------+----------------+-------------+ | 4 | ALTER_TENANT_PRIMARY_ZONE | SUCCESS | 0 | 100 | 2024-12-18 17:26:00.069089 | 2024-12-18 17:26:20.919021 | 1004 | ALTER TENANT mysql001 PRIMARY_ZONE='zone1,zone2' | FROM: 'zone1;zone2', TO: 'zone1,zone2' | 172.xx.xxx.xx | 2882 | | 5 | ALTER_TENANT_PRIMARY_ZONE | SUCCESS | 0 | 100 | 2024-12-18 17:41:44.412459 | 2024-12-18 17:41:54.965873 | 1004 | ALTER TENANT mysql001 PRIMARY_ZONE='zone1;zone2' | FROM: 'zone1,zone2', TO: 'zone1;zone2' | 172.xx.xxx.xx | 2882 | +--------+---------------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+--------------------------------------------------+------------------------------------------+----------------+-------------+ 2 rows in setIn the query result, find the corresponding job record based on the following fields:
START_TIME: the time when the job was initiated.SQL_TEXT: the SQL statement corresponding to the job.EXTRA_INFO: the primary zone information before and after the change.
If the value of
JOB_STATUSisSUCCESSin the corresponding job record, the job for reducing primary zones is successfully executed.For more information about the fields in the
DBA_OB_TENANT_JOBSview, see DBA_OB_TENANT_JOBS.Query the basic information of the
mysql001tenant. ThePRIMARY_ZONEattribute changes fromzone1,zone2tozone1;zone2, indicating that the modification is successful.obclient> SELECT PRIMARY_ZONE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_ID = 1004;The query result is as follows:
+--------------+ | PRIMARY_ZONE | +--------------+ | zone1;zone2 | +--------------+ 1 row in setQuery the distribution of the leader replica locations of the
mysql001tenant again.obclient> SELECT TENANT_ID, LS_ID, SVR_IP, SVR_PORT, ZONE, ROLE, REPLICA_TYPE FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE TENANT_ID = 1004 AND LS_ID != 1 AND ROLE = 'LEADER';The query result is as follows:
+-----------+-------+---------------+----------+--------+--------+--------------+ | TENANT_ID | LS_ID | SVR_IP | SVR_PORT | ZONE | ROLE | REPLICA_TYPE | +-----------+-------+---------------+----------+--------+--------+--------------+ | 1004 | 1001 | 172.xx.xxx.xx | 2882 | zone1 | LEADER | FULL | | 1004 | 1012 | 172.xx.xxx.xx | 2882 | zone1 | LEADER | FULL | +-----------+-------+---------------+----------+--------+--------+--------------+ 2 rows in setAs shown in the query result, tenant
mysql001has 2 leader replicas providing services in zone1.
The preceding example shows that before the change, tenant mysql001 has 2 zones with the highest PRIMARY_ZONE priority and 4 leader replicas providing services. After the change, the number of zones with the highest PRIMARY_ZONE priority decreases from 2 to 1, and the number of leader replicas providing services decreases from 4 to 2. The compute nodes are reduced, enabling the tenant to scale in.