If OceanBase Diagnostic Tool (obdiag) is independently deployed, you can use the obdiag rca commands to analyze the diagnostic information of OceanBase Database. These commands allow you to analyze exceptions of OceanBase Database to identify the causes that lead to these exceptions.
If your OceanBase cluster is deployed by using OceanBase Deployer (obd), you can run obdiag commands on obd to collect diagnostic information of the cluster. For more information about the commands, see obdiag commands.
Note: A large amount of internal table data is required for root cause analysis. Therefore, the tenant_sys user in the configuration file of obdiag must have the read privilege on tables and views in OceanBase Database.
Root cause analysis commands
# View supported root cause analysis scenarios
obdiag rca list
# Perform root cause analysis based on scenarios
obdiag rca run --scene=<scene_name>
Valid values of the scene_name parameter are as follows:
- disconnection: diagnoses disconnections based on the logs of OceanBase Database Proxy (ODP).
* major_hold: diagnoses stuck major compactions. lock_conflict: diagnoses lock contention.ddl_disk_full: diganoses the error of insufficient disk space reported during the execution of a DDL statement.transaction_disconnection: diagnoses transaction disconnections.log_error: troubleshoots log-related exceptions. Currently, exceptions where no leader is present are supported.transaction_not_ending: troubleshoots the issue where a transaction remains unterminated. At present, root cause analysis for this scenario is complex.transaction_rollback: diagnoses transaction rollback errors.transaction_execute_timeout: diagnoses timeout errors of transactions.clog_disk_full: diagnoses the issue where the clog disk space is exhausted.transaction_wait_timeout: diagnoses transaction wait timeout errors.transaction_other_error: diagnoses other transaction errors, such as errors corresponding to errors codes -4030, -4121, -4122, -4124, and -4019.
obdiag rca list
You can run this command to query the scenarios supported for root cause analysis.
obdiag rca run
You can run this command to perform root cause analysis for a specified scenario.
obdiag rca run --scene=<scene_name>
The following table describes the options.
Option |
Required? |
Data type |
Default value |
Description |
|---|---|---|---|---|
| --scene | No | string | Empty | The scenario for which root cause analysis is to be performed. You can run the obd obdiag rca list command to view the list of supported scenarios. |
| --store_dir | No | string | ./rca |
The path where the root cause analysis results are stored. |
| --input_parameters | No | string | Empty | The input environment variables. You can run the obd obdiag rca list command for help information. |
For example, you can run the following command to analyze a disconnection:
obdiag rca run --scene=disconnection
The output is as follows:
+-----------------------------------------------------------------------------------------------------------+
| record |
+------+----------------------------------------------------------------------------------------------------+
| step | info |
+------+----------------------------------------------------------------------------------------------------+
| 1 | node:xxx.xxx.xxx obproxy_diagnosis_log: [2024-01-18 17:48:37.667014] [23173][Y0-00007FAA5183E710] |
| | [CONNECTION](trace_type="CLIENT_VC_TRACE", connection_diagnosis={cs_id:1065, ss_id:4559, |
| | proxy_session_id:837192278409543969, server_session_id:3221810838, |
| | client_addr:" xxx.xxx.xxx.xxx:xxxx", server_addr:" xxx.xxx.xxx.xxx:2883", cluster_name:" obcluster", |
| | tenant_name:" sys", user_name:" root", error_code: -10010, error_msg:" An unexpected connection event |
| | received from client while obproxy reading request", request_cmd:" COM_SLEEP", sql_cmd:" COM_END", |
| | req_total_time(us):5315316}{vc_event:" VC_EVENT_EOS", user_sql:" "}) |
| 2 | cs_id:1065, server_session_id:3221810838 |
| 3 | trace_type:CLIENT_VC_TRACE |
| 4 | error_code: -10010 |
+------+----------------------------------------------------------------------------------------------------+
The suggest: Need client cooperation for diagnosis