After adaptive compaction is enabled, the system collects and analyzes real-time user query and write information to identify scenarios that may cause query slowdowns. It then adaptively schedules major compaction tasks to resolve these slow queries imperceptibly to users. During adaptive compaction, the system collects statistics and schedules major compactions at the partition level.
Using the adaptive compaction feature can help solve the following problems:
Prevent some buffer table issues.
Optimize query performance in data import scenarios.
Identify and avoid certain slow query scenarios.
Trigger rules
When a partition leader meets specific conditions, the system automatically schedules adaptive compaction for that partition and its follower replicas. The trigger rules for adaptive compaction are as follows:
The system analyzes the partition's statistics over a period to determine the number of queries and minor compactions, as well as the proportion of inserted rows in incremental data, to identify if it is a data import scenario. If so, adaptive compaction is triggered for the partition.
The system analyzes the partition's statistics over a period to determine the number of minor compactions, as well as the proportion of updated and deleted rows in incremental data, to identify if it is a data import scenario. If so, adaptive compaction is triggered for the partition.
The system counts the total number of incremental data rows in the partition to determine if it is a continuous write scenario. If so, adaptive compaction is triggered for the partition.
The system calculates the ratio of scanned data volume to actual effective data to determine if queries against the partition are slow. If so, adaptive compaction is triggered for the partition.
Considerations
Adaptive compaction can effectively reduce slow queries. If your business requires high query performance and there are scenarios supported by adaptive compaction, it is recommended not to disable this feature.
In OceanBase AI Database, parameters whose names start with an underscore (_) are hidden parameters, intended only for use by developers during troubleshooting or emergency maintenance. If you need to disable adaptive compaction, contact technical support for assistance.
Step 1: Enable adaptive compaction
The adaptive compaction feature is controlled by the tenant-level hidden parameter _enable_adaptive_compaction, which is enabled by default.
Log in to the
systenant of the cluster as therootuser.Query the value of the tenant-level hidden parameter
_enable_adaptive_compaction.obclient [oceanbase]> SELECT * FROM oceanbase.GV$OB_PARAMETERS WHERE NAME LIKE '_enable_adaptive_compaction';The query result is as follows:
+----------------+----------+-------+--------+-----------+-----------------------------+-----------+-------+---------------------------------------------------------------------------------+---------+-------------------+---------------+-----------+ | SVR_IP | SVR_PORT | ZONE | SCOPE | TENANT_ID | NAME | DATA_TYPE | VALUE | INFO | SECTION | EDIT_LEVEL | DEFAULT_VALUE | ISDEFAULT | +----------------+----------+-------+--------+-----------+-----------------------------+-----------+-------+---------------------------------------------------------------------------------+---------+-------------------+---------------+-----------+ | 172.xx.xxx.xxx | 2882 | zone1 | TENANT | 1 | _enable_adaptive_compaction | BOOL | True | specifies whether allow adaptive compaction schedule and information collection | TENANT | DYNAMIC_EFFECTIVE | True | YES | | 172.xx.xxx.xxx | 2882 | zone1 | TENANT | 1001 | _enable_adaptive_compaction | BOOL | True | specifies whether allow adaptive compaction schedule and information collection | TENANT | DYNAMIC_EFFECTIVE | True | YES | | 172.xx.xxx.xxx | 2882 | zone1 | TENANT | 1002 | _enable_adaptive_compaction | BOOL | True | specifies whether allow adaptive compaction schedule and information collection | TENANT | DYNAMIC_EFFECTIVE | True | YES | | 172.xx.xxx.xxx | 2882 | zone1 | TENANT | 1003 | _enable_adaptive_compaction | BOOL | True | specifies whether allow adaptive compaction schedule and information collection | TENANT | DYNAMIC_EFFECTIVE | True | YES | | 172.xx.xxx.xxx | 2882 | zone1 | TENANT | 1004 | _enable_adaptive_compaction | BOOL | True | specifies whether allow adaptive compaction schedule and information collection | TENANT | DYNAMIC_EFFECTIVE | True | YES | +----------------+----------+-------+--------+-----------+-----------------------------+-----------+-------+---------------------------------------------------------------------------------+---------+-------------------+---------------+-----------+ 5 rows in setIn the query result, if the value in the
VALUEcolumn isTrue, adaptive compaction is enabled.If adaptive compaction is not enabled, execute the following statement to enable it.
obclient [oceanbase]> ALTER SYSTEM SET _enable_adaptive_compaction = True TENANT = tenant_name;Notice
Since adaptive compaction scheduling tasks consume some system resources, to reduce resource usage, you can disable adaptive compaction scheduling for tables other than those in extreme mode by setting the tenant-level hidden parameter
_enable_adaptive_compactiontoFalse.
(Optional) Step 2: Specify the adaptive compaction strategy using the table_mode table option
After adaptive compaction is enabled, the table option table_mode is supported to more flexibly address performance degradation caused by buffer tables. You can set different table_mode values for each table to specify distinct fast-freeze and adaptive compaction strategies to combat buffer table issues.
You can specify an adaptive compaction strategy for a different mode using the table_mode table option when creating or modifying a table. The table_mode table option provides the following five modes to correspond to different compaction strategies.
Mode |
Probability of Triggering a Major Compaction After Minor Compaction |
|---|---|
| normal (default) | Extremely low |
| queuing | Low |
| moderate | Medium |
| super | High |
| extreme | Very High |
If you are familiar with your business, you can set an appropriate mode for a table during creation based on your business requirements. For example, create a table in queuing mode, as shown below.
obclient> CREATE TABLE tbl1 (c1 int, c2 double) table_mode = 'queuing';
In the preceding statement, if the table_mode value is not explicitly specified, the system creates a table in the default normal mode.
After creating a table, if you observe severe read amplification and buffer table phenomena in a particular table, you can manually modify its table_mode based on your business needs to trigger compactions more frequently and accelerate queries. However, since a higher compaction frequency consumes more computational resources, it is recommended to adjust the table_mode value step by step from low to high based on the frequency of compaction triggers. For example, start by adjusting from normal to queuing, observe for a period, and then adjust from queuing to moderate. An example of the adjustment is shown below.
obclient> ALTER TABLE tbl1 SET table_mode = 'moderate';
After the table modification statement executes successfully, the system will have a delay (about 2 minutes). Once the modification takes effect, the system will schedule adaptive compactions in the configured mode to resolve the buffer table phenomenon.