View end-to-end ODC-related logs

This topic describes how to view end-to-end ODC-related logs, which are mainly logs related to SQL execution links, including logs on the ODC, OBServer, and ODP nodes.

View ODC logs

For more information about logs for different ODC versions, see the following topics:

• View web ODC logs

• View client ODC logs

View OBServer logs

OBServer logs are stored in the log path under the installation directory of the OBServer node and include the observer.log, rootservice.log, and election.log types. These log types respectively correspond to the observer logs, RootService logs, and election logs.

Each log type can be further divided into subtypes by file name. For example, logs of the observer.log type can be divided to the following subtypes:

• observer.log

• observer.log.20210901123456

• observer.log.wf

• observer.log.wf.20210901123456

When the size of the observer.log file reaches 256 MB, the log file is renamed as the second type, suffixed with a timestamp. For more information about wf logs, see the description of the enable_syslog_wf parameter.

For more information about OBServer logs, see OceanBase Database logs.

OBServer log format

• Here is a sample log:

  [2021-09-01 11:31:18.605433] INFO [STORAGE] ob_pg_sstable_garbage_collector.cpp:170 [38715][0][Y0-0000000000000000-0-0] [lt=15] [dc=0] do one gc free sstable by queue(ret=0, free sstable cnt=0)

• The following log format applies:

  [time] log level [module] file: line number [thread id] [0] [trace_id] [lt=last log elapsed time (in microseconds)] [number of discarded logs]

View ODP logs

Requests with execution errors, including ODP errors and ODP errors, are printed to obproxy_error.log.

For more information about ODP logs, see OceanBase Database logs and monitoring.

Concatenating SQL logs by trace

In general, ODP does not record SQL statements that are executed correctly and records slow SQL statements and SQL statements with execution errors in obproxy_digest.log and obproxy_error.log respectively.

1. Query the SQL statements that are sent to ODP in ODC logs:

   grep your-sql /opt/odc/log/odc.log | tail -20
   

   Notice

   ODC may rewrite some SQL statements. Therefore, we recommend that you search for an SQL statement by keyword.

2. After you obtain the actually executed SQL statement, query the logs in ODP.

   grep real-sql obproxy_error.log | tail -20
   

   Notice

   If ODP is deployed in multi-node mode, you must retrieve the logs of each ODP.

   Here is a sample log:

   # ODP log
   [2022-02-24 14:26:38.781097] WARN  [PROXY.SM] set_client_abort (ob_mysql_sm.cpp:4790) [210278][Y0-7FAF8FB16890] [lt=7] [dc=0] client will abort soon(sm_id=33948448, cs_id=2007744, proxy_sessid=41059, ss_id=3619728, server_sessid=3221776979, client_ip={xx.x.xxx.xxx:xxxxx}, server_ip={xx.x.xxx.xx:xxxx}, cluster_name=sunshine_obcluster1, tenant_name=traveldb, user_name=travel, db=TRAVEL, event="VC_EVENT_EOS", request_cmd="Quit", sql_cmd="Query", sql=)
   

3. After you obtain server_sessid in the ODP log, query the log corresponding to server_sessid in the OBServer log. Search keyword: sessid=xxx. Here is a sample log:

   # OBServer log
   [2022-02-24 14:26:38.781201] WARN  [SQL.SESSION] set_session_state (ob_basic_session_info.cpp:4184) [646225][3336][YB420A089429-0005CB40678E174E] [lt=8] [dc=0] session is killed(ret=-5066, sessid_=3221776979, proxy_sessid_=41059, version_=0, state=2)