If OceanBase Diagnostic Tool (obdiag) is independently deployed, you can use the obdiag rca commands to analyze the diagnostic information of OceanBase Database. These commands allow you to analyze exceptions of OceanBase Database to identify the causes that lead to these exceptions.
If your OceanBase cluster is deployed by using OceanBase Deployer (obd), you can run obdiag commands on obd to collect diagnostic information of the cluster. For more information about the commands, see obdiag commands.
Notice: A large amount of internal table data is required for root cause analysis. Therefore, the tenant_sys user in the configuration file of obdiag must have the read permission on tables and views in the oceanbase database.
Root cause analysis commands
# View supported root cause analysis scenarios
obdiag rca list
# Perform root cause analysis based on scenarios
obdiag rca run --scene=<scene_name>
Valid values of the scene_name parameter are as follows:
disconnection: diagnoses disconnections based on the logs of OceanBase Database Proxy (ODP).major_hold: diagnoses stuck major compactions.lock_conflict: diagnoses lock contention.ddl_disk_full: diagnoses the error of insufficient disk space reported during the execution of a DDL statement.ddl_failure: diagnoses failed DDL statements.transaction_disconnection: diagnoses transaction disconnections.log_error: diagnoses log-related exceptions. At present, exceptions where no leader is present are supported.transaction_not_ending: diagnoses the issue where a transaction remains unterminated. At present, root cause analysis for this scenario is complex.transaction_rollback: diagnoses transaction rollback errors.transaction_execute_timeout: diagnoses timeout errors of transactions.clog_disk_full: diagnoses the issue where the clog disk space is exhausted.transaction_wait_timeout: diagnoses transaction wait timeout errors.transaction_other_error: diagnoses other transaction errors, such as errors corresponding to errors codes-4030,-4121,-4122,-4124, and-4019.index_ddl_error: diagnoses index creation errors.
obdiag rca list
You can run this command to query the scenarios supported for root cause analysis.
obdiag rca run
You can run this command to perform root cause analysis for a specified scenario.
obdiag rca run --scene=<scene_name>
The following table describes the options.
Option |
Required |
Data type |
Default value |
Description |
|---|---|---|---|---|
| --scene | No | string | Empty | The scenario for which root cause analysis is to be performed. You can run the obdiag rca list command to view the list of supported scenarios. |
| --store_dir | No | string | ./rca |
The path where the root cause analysis results are stored. |
| --env | No | string | Empty | The input environment variables. Each variable must be preceded by --env, such as --env env1=value1 --env env2=value2. For more information, see the topics about specific scenarios. |
For example, you can run the following command to analyze a disconnection:
obdiag rca run --scene=disconnection
The output is as follows:
+-----------------------------------------------------------------------------------------------------------+
| record |
+------+----------------------------------------------------------------------------------------------------+
| step | info |
+------+----------------------------------------------------------------------------------------------------+
| 1 | node:xxx.xxx.xxx obproxy_diagnosis_log:[2024-01-18 17:48:37.667014] [23173][Y0-00007FAA5183E710] |
| | [CONNECTION](trace_type="CLIENT_VC_TRACE", connection_diagnosis={cs_id:1065, ss_id:4559, |
| | proxy_session_id:837192278409543969, server_session_id:3221810838, |
| | client_addr:"xxx.xxx.xxx.xxx:xxxx", server_addr:"xxx.xxx.xxx.xxx:2883", cluster_name:"obcluster", |
| | tenant_name:"sys", user_name:"root", error_code:-10010, error_msg:"An unexpected connection event |
| | received from client while obproxy reading request", request_cmd:"COM_SLEEP", sql_cmd:"COM_END", |
| | req_total_time(us):5315316}{vc_event:"VC_EVENT_EOS", user_sql:""}) |
| 2 | cs_id:1065, server_session_id:3221810838 |
| 3 | trace_type:CLIENT_VC_TRACE |
| 4 | error_code:-10010 |
+------+----------------------------------------------------------------------------------------------------+
The suggest: Need client cooperation for diagnosis