Alert description
This alert is triggered when an OBProxy server generates a core dump. The alert will no longer be triggered 20 minutes after the core dump occurs.
Alert principle
Parameter |
Value |
|---|---|
| Monitoring metric | obproxy_core_dump_seconds |
| Monitoring expression | max(obproxy_coredump_time_seconds{@LABELS}) by (@GBLABELS) |
| Sampling interval | 60 seconds |
Rule information
Monitoring expression |
Description |
Default threshold |
Detection cycle |
Elimination cycle |
|---|---|---|---|---|
| obproxy_core_dump_seconds < 1200 and obproxy_core_dump_seconds > 0 | The duration of the most recent core dump for OBProxy | 0 | 10 seconds | 5 minutes |
Alert information
Alert trigger method |
Alert level |
Scope |
|---|---|---|
| Based on the expression of the monitoring metric | Severe | Host |
Alert template
Alert overview
- Template: ${alarm_target} ${alarm_name}
- Example: alarm_template_id=2:obproxy_cluster=TEST-2000005:host=xxx.xxx.xxx.xxx OBProxy server Core Dump
Alert details
- Template: Cluster: ${obproxy_cluster}, Host: ${host}, Alert: ${value_shown} before the OBProxy server Core Dump.
- Example: Cluster: TEST, Host: xxx.xxx.xxx.xxx, Alert: 19 minutes and 50.54 seconds before the OBProxy server Core Dump.
Alert recovery
- Template: Alert: ${alarm_name}, Time since the most recent core dump for OBProxy: ${value_shown}
- Example: Alert: OBProxy server Core Dump, Time since the most recent Core Dump for OBProxy: 0 seconds
Impact on the system
A core dump occurs when a process encounters an unexpected error, and the core dump is used to save the environment at the time of the exception for post-event analysis. After the core dump, the process is restarted, which is a protective measure for the system. The impact of unexpected situations on the system varies depending on the specific circumstances.
Possible causes
- The program has an unexpected error.
- The OBProxy version is too low, which may have design limitations, such as when the size of a MySQL packet exceeds 16 MB in a lower version, it will trigger a core dump.
Resolution
- Obtain the core dump file and contact OBProxy technical support for analysis. The default path for the core dump file is
/home/admin/logs/obproxy/minidumpin the OBProxy runtime directory, with the file extension.dmp. - If the memory allocated to OBProxy is excessively large, it may result in an extended core dump process. For example, if the default memory is 2 GB and the allocated memory far exceeds the default value, an emergency operation to restart the OBProxy process (ignoring the core dump) should be performed to prevent the issue from worsening. Additionally, consider upgrading to a newer version of OBProxy, which may include optimized memory management.