Node network card overload

The high load on the network card of the OBServer node is generally related to background tasks such as batch processing and replication migration. The approach to handling this issue is similar to dealing with high node disk I/O. It usually involves downgrading tasks and adjusting migration/replication-related parameters. This topic provides information on how to address this issue in emergency situations.

Emergency handling procedure

1. Pause running backup tasks.

   You can use OCP to check if the current node is performing backups. If so, pausing the backups can alleviate the I/O pressure.

2. Pause running data import/export tasks.

   You can use OMS to check if the current node is involved in data migration tasks. If it is, you can pause them as needed to relieve the I/O pressure before resuming the tasks. Only running data migration projects support pause and resume operations. After pausing a data migration project, OMS will terminate the running subtasks in the orchestration task list and automatically terminate the scheduling of subsequent subtasks.

   You can manually continue the paused or failed data migration projects later. OMS will start executing each subtask one by one, starting from the first pending task in the orchestration task list.

3. Reduce the network bandwidth of background tasks.

   alter system set sys_bkgd_net_percentage=30;  -- Default value: 60.
   

4. Enable clog transmission compression.

   You can enable log transmission compression by setting the clog_transport_compress_all parameter to True. The setting takes effect immediately without the need for a restart.

   ALTER SYSTEM SET log_transport_compress_all = TRUE ;
   

5. Throttle the execution of high-load SQL queries.

   If a particular high-load SQL query is identified, you can limit the concurrency of the SQL query by adding the max_concurrent hint to its execution plan. This achieves throttling the SQL query.

   CREATE OUTLINE outline_name ON sql_id USING HINT /*+max_concurrent(1)*/;