observer.cluster_down|V2.6.0|OceanBase Diagnostic Tool |docs|OceanBase Documentations

If OceanBase Diagnostic Tool (obdiag) is independently deployed, you can specify the observer.cluster_down scenario in the obdiag gather scene run command to collect the information required for analyzing cluster downtime. The command helps address the pain point in collecting information on distributed nodes.

Syntax

obdiag gather scene run --scene=observer.cluster_down [options]

The following table describes the options.

Option	Required?	Data type	Default value	Description
--scene	Yes	string	Empty	The name of the scenario. You can run the `obd obdiag gather scene list` command to view the scenarios supported by the current version.
--from	No	string	Empty	The start time of log collection in the `yyyy-mm-dd hh:mm:ss` format.
--to	No	string	Empty	The end time of log collection in the `yyyy-mm-dd hh:mm:ss` format.
--since	No	string	Empty	The most recent period of time for log collection, in the format of `<n> <m\|h\|d>`, where `n` specifies the time value, `m` represents "minute", `h` represents "hour", and `d` represents "day". For example, `30m` specifies to collect logs of the last 30 minutes.
--store_dir	No	string	The current path where the command is executed	The local path where the results are stored.
--temp_dir	No	string	`/tmp`	The directory where temporary files generated by the remote node during log collection are stored.
-c	No	string	`~/.obdiag/config.yml`	The path of the configuration file.
--inner_config	No	string	Empty	The configurations of obdiag.
--config	No	string	Empty	The configurations of the cluster diagnosed by obdiag, in the format of `--config key1=value1 --config key2=value2`. Note For information about the parameters supported by this option, see Configure obdiag.

Examples

Method 1: Use the command out-of-the-box without a configuration file

# Collect information for cluster downtime. If you do not specify any other options, the command collects the information of the last 30 minutes.
obdiag gather scene run --scene=observer.cluster_down \
    --config obcluster.servers.nodes[0].ip=xx.xx.xx.1 \
    --config obcluster.servers.nodes[1].ip=xx.xx.xx.xx.2 \
    --config obcluster.servers.nodes[2].ip=xx.xx.xx.xx.3 \
    --config obcluster.servers.global.ssh_username=test \
    --config obcluster.servers.global.ssh_password=****** \
    --config obcluster.servers.global.home_path=/home/admin/oceanbase

# Use the `--since` option to specify the time range of information to be collected for cluster downtime.
obdiag gather scene run --scene=observer.cluster_down --since 30m \
    --config obcluster.servers.nodes[0].ip=xx.xx.xx.1 \
    --config obcluster.servers.nodes[1].ip=xx.xx.xx.xx.2 \
    --config obcluster.servers.nodes[2].ip=xx.xx.xx.xx.3 \
    --config obcluster.servers.global.ssh_username=test \
    --config obcluster.servers.global.ssh_password=****** \
    --config obcluster.servers.global.home_path=/home/admin/oceanbase

# Use the `--from` and `--to` options to specify the time range of cluster downtime.
obdiag gather scene run --scene=observer.cluster_down --from "2022-06-30 16:25:00" --to "2022-06-30 18:30:00" \
    --config obcluster.servers.nodes[0].ip=xx.xx.xx.1 \
    --config obcluster.servers.nodes[1].ip=xx.xx.xx.xx.2 \
    --config obcluster.servers.nodes[2].ip=xx.xx.xx.xx.3 \
    --config obcluster.servers.global.ssh_username=test \
    --config obcluster.servers.global.ssh_password=****** \
    --config obcluster.servers.global.home_path=/home/admin/oceanbase

Method 2: Use the command with a configuration file

Before you run the command, make sure that you have configured the logon information of the target nodes in the config.yml configuration file of obdiag. For more information, see Configure obdiag.

# Collect information for cluster downtime. If you do not specify any other options, the command collects the information of the last 30 minutes.
obdiag gather scene run --scene=observer.cluster_down

# Use the `--since` option to specify the time range of information to be collected for cluster downtime.
obdiag gather scene run --scene=observer.cluster_down --since 30m

# Use the `--from` and `--to` options to specify the time range of cluster downtime.
obdiag gather scene run --scene=observer.cluster_down --from "2022-06-30 16:25:00" --to "2022-06-30 18:30:00"

Results

The command generates a folder in the specified directory. A sample folder name is obdiag_gather_pack_20240808105846.

#tree
.
├── ob_log_remote_xx.xx.xx.xx_20240808201458_20240808204558.zip # The logs of OBServer nodes.
├── result_summary.txt
├── shell_result.txt # The command execution results required for analyzing cluster downtime.
└── sysstat_xx.xx.xx.xx_20240808204521.zip # The host information required for analyzing cluster downtime.

Cases

When a cluster is down, you can run the following command to collect the information required for diagnostics. Then, you can post the information on the OceanBase community forum or send it to OceanBase Technical Support.

obdiag gather scene run --scene=observer.cluster_down \
    --config obcluster.servers.nodes[0].ip=xx.xx.xx.1 \
    --config obcluster.servers.nodes[1].ip=xx.xx.xx.xx.2 \
    --config obcluster.servers.nodes[2].ip=xx.xx.xx.xx.3 \
    --config obcluster.servers.global.ssh_username=test \
    --config obcluster.servers.global.ssh_password=****** \
    --config obcluster.servers.global.home_path=/home/admin/oceanbase