This topic describes how to reduce the number of replicas for a log stream of a tenant.
Reduce replicas by modifying locality
You can reduce the number of log stream replicas for a tenant by modifying the locality of the tenant in the cluster. For more information about how to modify the locality, see Modify locality.
Prerequisites
Before you modify the locality of a tenant, verify the resource information of each node in the target zone. If the nodes in the target zone do not have sufficient resources to accommodate the resource units required by the tenant, the locality cannot be changed.
For more information about how to view the resource information of each node in a zone, see View tenant and resource information.
Considerations
The locality of a tenant must be specified.
When the locality of a tenant changes, the distribution of table replicas in the tenant also changes.
When a tenant locality modification is not finished, the next tenant locality modification cannot be performed.
After you reduce replicas from a tenant, you can remove resource pools from the tenant if you want to release resources.
Assume that the
z1andz2zones are configured for a tenant and you want to remove thez2zone. The following scenarios exist:The tenant has an independent resource pool in
z2. For example, the tenant has two resource pools:resource_pool1andresource_pool2.resource_pool2belongs toz2. After you reduce the replicas inz2, you must removeresource_pool2from the resource pool list of the tenant. Then you can drop the resource pool to release the resources. For more information, see Drop a resource pool.The tenant has no independent resource pool in
z2. For example, the resource poolresource_pool1of the tenant belongs toz1. After you reduce the replicas inz2, you must removez2from the zone list ofresource_pool1to release the resources. For more information about how to modify the zone list of a resource pool, see Modify attributes of a resource pool.
Procedure
The following example describes how to modify the locality of the mysql001 tenant from FULL{1}@zone1, FULL{1}@zone2 to FULL{1}@zone1 and reduce the number of replicas for the tenant.
Log in to the
systenant of the cluster as therootuser.obclient -h172.30.xxx.xxx -P2883 -uroot@sys#obdemo -pxxxx -AAccess the database named
oceanbase.obclient(root@sys)[(none)]> USE oceanbase;Query the current locality of the
mysql001tenant.obclient(root@sys)[oceanbase]> SELECT TENANT_ID,TENANT_NAME,TENANT_TYPE,PRIMARY_ZONE,LOCALITY FROM oceanbase.DBA_OB_TENANTS;The query result is as follows:
+-----------+-------------+-------------+-------------------+---------------------------------------------+ | TENANT_ID | TENANT_NAME | TENANT_TYPE | PRIMARY_ZONE | LOCALITY | +-----------+-------------+-------------+-------------------+---------------------------------------------+ | 1 | sys | SYS | zone1;zone2 | FULL{1}@zone1, FULL{1}@zone2 | | 1001 | META$1002 | META | zone1;zone2 | FULL{1}@zone1, FULL{1}@zone2 | | 1002 | mysql001 | USER | zone1;zone2 | FULL{1}@zone1, FULL{1}@zone2 | +-----------+-------------+-------------+-------------------+---------------------------------------------+ 3 rows in setYou can see that the
mysql001tenant has one full-featured replica in each ofzone1andzone2.Modify the locality of the
mysql001tenant.obclient(root@sys)[oceanbase]> ALTER TENANT mysql001 locality="FULL{1}@zone1";View the execution status of the locality change job.
obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.DBA_OB_TENANT_JOBS WHERE JOB_TYPE = 'ALTER_TENANT_LOCALITY';The query result is as follows:
+--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+ | JOB_ID | JOB_TYPE | JOB_STATUS | RESULT_CODE | PROGRESS | START_TIME | MODIFY_TIME | TENANT_ID | SQL_TEXT | EXTRA_INFO | RS_SVR_IP | RS_SVR_PORT | +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+ | 1 | ALTER_TENANT_LOCALITY | SUCCESS | 0 | 100 | 2023-01-05 19:33:46.730319 | 2023-01-05 19:33:46.920111 | 1002 | ALTER TENANT mysql001 locality='FULL{1}@zone1' | FULL{1}@zone1, FULL{1}@zone2 | xx.xx.xx.xx | 2882 | +--------+-----------------------+------------+-------------+----------+----------------------------+----------------------------+-----------+------------------------------------------------------------------------------+---------------------------------------------+----------------+-------------+ 1 row in setIf the value of
JOB_STATUSisSUCCESS, the locality change job is successful.Query the locality of the
mysql001tenant after the modification.obclient(root@sys)[oceanbase]> SELECT TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY FROM oceanbase.DBA_OB_TENANTS;The query result is as follows:
+-----------+-------------+-------------+-------------------+---------------------------------------------+ | TENANT_ID | TENANT_NAME | TENANT_TYPE | PRIMARY_ZONE | LOCALITY | +-----------+-------------+-------------+-------------------+---------------------------------------------+ | 1 | sys | SYS | zone1;zone2 | FULL{1}@zone1, FULL{1}@zone2 | | 1001 | META$1002 | META | zone1 | FULL{1}@zone1 | | 1002 | mysql001 | USER | zone1 | FULL{1}@zone1 | +-----------+-------------+-------------+-------------------+---------------------------------------------+ 3 rows in setThe query results in step 3 and step 6 show that the locality of the
mysql001tenant is changed fromFULL{1}@zone1, FULL{1}@zone2toFULL{1}@zone1, and the number of replicas for themysql001tenant is reduced.
Manually remove a replica
When you reduce the number of replicas by modifying the locality of the tenant, the system automatically removes the redundant replicas. However, if the system does not properly remove replicas due to exceptions, you can execute the ALTER SYSTEM REMOVE REPLICA statement to remove the redundant log stream replicas.
Limitations
You can manage log streams of all tenants in the
systenant but manage only log streams of the current tenant in a user tenant.For the same log stream of a tenant, except that multiple replica migration tasks can be executed in parallel, only one other disaster recovery task (such as adding a replica, deleting a replica, or modifying the quorum size of a log stream) can be executed at a time.
The
systenant can query theCDB_OB_LS_REPLICA_TASKSview to check whether a disaster recovery task is in progress. A user tenant can query theDBA_OB_LS_REPLICA_TASKSview to check whether a disaster recovery task is in progress.
Prerequisites
Before you remove a replica, make sure that you have the
ALTER SYSTEMprivilege. Otherwise, you cannot execute theALTER SYSTEM REMOVE REPLICAstatement.Before you query a view, make sure that you have the
SELECTprivilege on the view. Otherwise, you cannot query the view.DBA_OB_TENANTSDBA_OB_LS/CDB_OB_LSDBA_OB_LS_LOCATIONS/CDB_OB_LS_LOCATIONS
Procedure
Assume that the locality of a tenant named tenant1 is F@zone1,F@zone3, and log stream 1001 of the tenant1 tenant has a full-featured replica in each of zone1, zone2, and zone3. To remove the replica in zone2, perform the following steps:
Log in to the target tenant.
Here is an example:
obclient -h172.30.xxx.xxx -P2883 -uroot@tenant1#obdemo -pxxxx -AFor more information about how to connect to a database, see Overview of connection methods.
Query the tenant ID and locality of the target tenant.
System tenant
obclient(root@sys)[oceanbase]> SELECT TENANT_NAME,TENANT_ID,LOCALITY FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_NAME='tenant1';User tenant
Execute the following statement:
obclient(root@tenant1)[oceanbase]> SELECT TENANT_NAME,TENANT_ID,LOCALITY FROM oceanbase.DBA_OB_TENANTS;Here is an example of the query result:
+-------------+-----------+------------------------------+ | TENANT_NAME | TENANT_ID | LOCALITY | +-------------+-----------+------------------------------+ | tenant1 | 1002 | FULL{1}@zone1, FULL{1}@zone3 | +-------------+-----------+------------------------------+ 1 row in setThe tenant ID is
1002.For more information about the fields in the
DBA_OB_TENANTSview, see DBA_OB_TENANTS.
Query all log streams of the target tenant.
System tenant
obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS WHERE TENANT_ID=1002;For more information about the fields in the
CDB_OB_LSview, see CDB_OB_LS.User tenant
Execute the following statement:
obclient(root@tenant1)[oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS;Here is an example of the query result:
+-------+--------+--------------+---------------+-------------+---------------------+----------+---------------------+---------------------+------+----------------+ | LS_ID | STATUS | PRIMARY_ZONE | UNIT_GROUP_ID | LS_GROUP_ID | CREATE_SCN | DROP_SCN | SYNC_SCN | READABLE_SCN | FLAG | UNIT_LIST | +-------+--------+--------------+---------------+-------------+---------------------+----------+---------------------+---------------------+------+----------------+ | 1 | NORMAL | zone1;zone3 | 0 | 0 | NULL | NULL | 1712136316868815003 | 1712136316868815003 | | | | 1001 | NORMAL | zone1;zone3 | 1001 | 1001 | 1711951923021132001 | NULL | 1712136316868815003 | 1712136316868815003 | | 1001,1002,1003 | +-------+--------+--------------+---------------+-------------+---------------------+----------+---------------------+---------------------+------+----------------+ 2 rows in setThe query result shows that the
tenant1tenant has log streams1and1001.For more information about the fields in the
DBA_OB_LSview, see DBA_OB_LS.
Query the replica distribution of log stream
1001based on the obtained log stream information.System tenant
obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE LS_ID=1001 AND TENANT_ID=1002;For more information about the fields in the
CDB_OB_LS_LOCATIONSview, see CDB_OB_LS_LOCATIONS.User tenant
Execute the following statement:
obclient(root@tenant1)[oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS_LOCATIONS WHERE LS_ID=1001;Here is an example of the query result:
+----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+----------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+ | CREATE_TIME | MODIFY_TIME | LS_ID | SVR_IP | SVR_PORT | SQL_PORT | ZONE | ROLE | MEMBER_LIST | PAXOS_REPLICA_NUMBER | REPLICA_TYPE | LEARNER_LIST | REBUILD | +----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+----------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+ | 2024-04-03 17:23:06.381521 | 2024-04-03 17:23:15.934957 | 1001 | 172.xx.xxx.192 | 2882 | 2881 | zone3 | FOLLOWER | NULL | NULL | FULL | | FALSE | | 2024-04-01 14:12:08.588944 | 2024-04-03 17:23:14.904070 | 1001 | 172.xx.xxx.212 | 2882 | 2881 | zone2 | FOLLOWER | NULL | NULL | FULL | | FALSE | | 2024-04-01 14:12:08.589084 | 2024-04-03 17:23:14.080074 | 1001 | 172.xx.xxx.226 | 2882 | 2881 | zone1 | LEADER | 172.xx.xxx.192:2882:1712136186291857,172.xx.xxx.212:2882:1,172.xx.xxx.226:2882:1 | 3 | FULL | | FALSE | +----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+----------------------------------------------------------------------------------+----------------------+--------------+--------------+---------+ 3 rows in setThe query result shows that log stream 1001 has a full-featured replica on each of the servers
172.xx.xxx.192:2882,172.xx.xxx.212:2882, and172.xx.xxx.226:2882. Based on the locality, you must remove the replica from the server172.xx.xxx.212:2882inzone2.For more information about the fields in the
DBA_OB_LS_LOCATIONSview, see DBA_OB_LS_LOCATIONS.
Execute the
ALTER SYSTEM REMOVE REPLICAstatement to remove the replica.The syntax is as follows:
ALTER SYSTEM REMOVE REPLICA LS [=] ls_id SERVER [=] 'svr_ip:svr_port' [PAXOS_REPLICA_NUM [=] paxos_replica_num] [TENANT [=] 'tenant_name'];The parameters in the syntax are described as follows:
ls_id: the ID of the log stream for which you want to remove the replica.svr_ip:svr_port: the IP address and port number of the server from which you want to remove the replica, for example,172.xx.xxx.212:2882.paxos_replica_num: the modified number of Paxos replicas of the log stream, which is the number of full-featured replicas specified in the locality of the tenant. A log stream supports a maximum of seven full-featured replicas.The value of the
paxos_replica_numparameter must meet the following conditions:The value of the
paxos_replica_numparameter must be greater than or equal to the number of members in theMEMBER_LISTof the leader.The absolute value of the variation of the
paxos_replica_numvalue is not greater than 1. In this example, the value of thepaxos_replica_numparameter is3, and you can change it only to2.
If any one of the preceding conditions is not met, the system will return an error. If this parameter is not explicitly specified, the default value is used.
tenant_name: the name of the target tenant. You can specify another tenant in thesystenant and specify only the current tenant in a user tenant. If this parameter is not explicitly specified, the name of the current tenant is used. You cannot use theall,all_user, orall_metaoption in this statement to specify all tenants, all user tenants, or all meta tenants.This statement removes only one replica at a time.
Here is an example: Remove the replica of log stream
1001on the server172.xx.xxx.212:2882and set thePAXOS_REPLICA_NUMparameter to2.obclient(root@tenant1)[oceanbase]> ALTER SYSTEM REMOVE REPLICA LS = 1001 SERVER = '172.xx.xxx.212:2882' PAXOS_REPLICA_NUM = 2;Query information about the log stream again to verify that one replica of the log stream is removed.
System tenant
obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS_LOCATIONS WHERE LS_ID=1001 AND TENANT_ID=1002;User tenant
Execute the following statement:
obclient(root@tenant1)[oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS_LOCATIONS WHERE LS_ID=1001;Here is an example of the query result:
+----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+------------------------------------------------------------+----------------------+--------------+--------------+---------+ | CREATE_TIME | MODIFY_TIME | LS_ID | SVR_IP | SVR_PORT | SQL_PORT | ZONE | ROLE | MEMBER_LIST | PAXOS_REPLICA_NUMBER | REPLICA_TYPE | LEARNER_LIST | REBUILD | +----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+------------------------------------------------------------+----------------------+--------------+--------------+---------+ | 2024-04-03 17:23:06.381521 | 2024-04-03 17:32:49.079886 | 1001 | 172.xx.xxx.192 | 2882 | 2881 | zone3 | FOLLOWER | NULL | NULL | FULL | | FALSE | | 2024-04-01 14:12:08.589084 | 2024-04-03 17:32:49.124409 | 1001 | 172.xx.xxx.226 | 2882 | 2881 | zone1 | LEADER | 172.xx.xxx.192:2882:1712136186291857,172.xx.xxx.226:2882:1 | 2 | FULL | | FALSE | +----------------------------+----------------------------+-------+----------------+----------+----------+-------+----------+------------------------------------------------------------+----------------------+--------------+--------------+---------+ 2 rows in setThe query result shows that log stream 1001 has a full-featured replica on each of the servers
172.xx.xxx.192:2882and172.xx.xxx.226:2882, and the replica on the server172.xx.xxx.212:2882has been removed.