The execution of current disaster recovery tasks is subject to a limitation: except for migration replica tasks, a tenant's log stream can only allow one replica task to be executed at a time. This means some urgent tasks may need to wait until the ongoing task is completed before they can be executed. If you need to prioritize an urgent task, you can use the ALTER SYSTEM CANCEL REPLICA TASK command to manually cancel the running replica task.
Limitations
The current version only supports canceling two types of replica tasks: supplement replica and migration replica tasks.
Note
In the current version, for a replica migration task caused by unit migration, if the destination side has no units left, the system automatically cancels the corresponding replica migration task; manual cancellation is not required.
The
systenant can cancel replica tasks for all tenants. User tenants can only cancel replica tasks for their own tenant.
Prerequisites
Before canceling a replica task, ensure the current user has the
ALTER SYSTEMprivilege; otherwise, theALTER SYSTEM CANCEL REPLICA TASKcommand cannot be executed.Before querying views, ensure the current user has the
SELECTprivilege on the following views; otherwise, you cannot query the relevant information.DBA_OB_LS_REPLICA_TASKS/CDB_OB_LS_REPLICA_TASKSDBA_OB_LS_REPLICA_TASK_HISTORY/CDB_OB_LS_REPLICA_TASK_HISTORY
Procedure
Assume that a tenant named tenant1 has an F replica of its log stream 1001 undergoing a migration task. At this moment, a failure occurs in the current cluster, and you urgently need to add an F replica on an available server to ensure high availability. However, because the data volume of the F replica being migrated is very large, the task takes a long time to complete, which prevents the urgent maintenance operation of adding the F replica from being executed in time. In this case, O&M personnel can manually cancel the disaster recovery task for the migrating F replica as described in this topic, so as to prioritize urgent maintenance operations.
Log in to the tenant corresponding to the cluster.
Here is an example:
obclient -h172.30.xxx.xxx -P2883 -uroot@tenant1#obdemo -pxxxx -AFor more information about how to connect to a database, see Overview of connection methods.
Query the currently running disaster recovery tasks.
System tenant
obclient [oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS_REPLICA_TASKS\GFor more information about the fields in the
CDB_OB_LS_REPLICA_TASKSview, see CDB_OB_LS_REPLICA_TASKS.User tenant
Execute the following statement:
obclient [oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS_REPLICA_TASKS\GHere is an example of the query result:
*************************** 1. row *************************** LS_ID: 1001 TASK_TYPE: MIGRATE REPLICA TASK_ID: Y13CE64586BD4-000610C5F3EDBBCB-0-0 TASK_STATUS: INPROGRESS PRIORITY: LOW TARGET_REPLICA_SVR_IP: 100.xx.xxx.002 TARGET_REPLICA_SVR_PORT: 5072 TARGET_PAXOS_REPLICA_NUMBER: 2 TARGET_REPLICA_TYPE: FULL SOURCE_REPLICA_SVR_IP: 100.xx.xxx.003 SOURCE_REPLICA_SVR_PORT: 5073 SOURCE_PAXOS_REPLICA_NUMBER: 2 SOURCE_REPLICA_TYPE: FULL DATA_SOURCE_SVR_IP: 100.xx.xxx.003 DATA_SOURCE_SVR_PORT: 5073 IS_MANUAL: FALSE TASK_EXEC_SVR_IP: 100.xx.xxx.002 TASK_EXEC_SVR_PORT: 5072 CREATE_TIME: 2024-02-07 15:23:04 START_TIME: 2024-02-07 15:23:04 MODIFY_TIME: 2024-02-07 15:23:04 COMMENT: migrate replica due to unit group not match CONFIG_VERSION: 1 row in setThe query result indicates that a disaster recovery task with
TASK_TYPEset toMIGRATE REPLICAis in progress, and itsTASK_IDis recorded.For more information about the fields in the
DBA_OB_LS_REPLICA_TASKSview, see DBA_OB_LS_REPLICA_TASKS.
Execute the command to cancel the replica task.
The statement is as follows:
ALTER SYSTEM CANCEL REPLICA TASK TASK_ID [=] 'task_id' [TENANT [=] 'tenant_name'];The statement is used as follows:
task_id: specifies theTASK_IDof the replica task to be canceled.tenant_name: specifies the tenant to be operated on. You can specify another tenant in thesystenant and specify only the current tenant in a user tenant. If you do not explicitly specify the tenant, the default tenant name is the current tenant. This statement does not support usingall,all_user, orall_metato specify all tenants, user tenants, or all Meta tenants.This statement can only cancel one disaster recovery task per tenant at a time.
Here is an example:
obclient> ALTER SYSTEM CANCEL REPLICA TASK TASK_ID = 'Y13CE64586BD4-000610C5F3EDBBCB-0-0';Query the cancellation result of the replica task.
System tenant
obclient [oceanbase]> SELECT * FROM oceanbase.CDB_OB_LS_REPLICA_TASK_HISTORY WHERE TASK_ID = 'Y13CE64586BD4-000610C5F3EDBBCB-0-0'\GFor more information about the fields in the
CDB_OB_LS_REPLICA_TASK_HISTORYview, see CDB_OB_LS_REPLICA_TASK_HISTORY.User tenant
Execute the following statement:
obclient [oceanbase]> SELECT * FROM oceanbase.DBA_OB_LS_REPLICA_TASK_HISTORY WHERE TASK_ID = 'Y13CE64586BD4-000610C5F3EDBBCB-0-0'\GHere is an example of the query result:
*************************** 1. row *************************** LS_ID: 1001 TASK_TYPE: MIGRATE REPLICA TASK_ID: Y13CE64586BD4-000610C5F3EDBBCB-0-0 TASK_STATUS: CANCELED PRIORITY: LOW TARGET_REPLICA_SVR_IP: 100.xx.xxx.002 TARGET_REPLICA_SVR_PORT: 5072 TARGET_PAXOS_REPLICA_NUMBER: 2 TARGET_REPLICA_TYPE: FULL SOURCE_REPLICA_SVR_IP: 100.xx.xxx.003 SOURCE_REPLICA_SVR_PORT: 5073 SOURCE_PAXOS_REPLICA_NUMBER: 2 SOURCE_REPLICA_TYPE: FULL DATA_SOURCE_SVR_IP: 100.xx.xxx.003 DATA_SOURCE_SVR_PORT: 5073 IS_MANUAL: FALSE TASK_EXEC_SVR_IP: 100.xx.xxx.002 TASK_EXEC_SVR_PORT: 5072 CREATE_TIME: 2024-02-07 15:54:35 START_TIME: 2024-02-07 15:54:31 MODIFY_TIME: 2024-02-07 15:54:35 FINISH_TIME: 2024-02-07 15:54:34 EXECUTE_RESULT: ret:-4072, OB_CANCELED; elapsed:4109659; comment:[storage] receive task reply from storage rpc; COMMENT: migrate replica due to unit not match CONFIG_VERSION: 1 row in setIf the value of
TASK_STATUSisCANCELED, the task has been canceled.For more information about the fields in the
DBA_OB_LS_REPLICA_TASK_HISTORYview, see DBA_OB_LS_REPLICA_TASK_HISTORY.