This topic describes the types, status, and compression algorithms of major compactions.
A major compaction merges dynamic and static data, which is a time-consuming process. When the incremental data generated by minor compactions accumulates to a certain level, a major freeze is performed to merge data into a new major version. The main difference between a major compaction and a minor compaction is that a major compaction merges the tenant's data with its corresponding static data at a unified snapshot point, ultimately forming a tenant-level snapshot.
Types of major compactions
Based on the amount of data involved, major compactions can be classified as follows:
Full compaction: Reads all static data and merges it with dynamic data to produce the final static data. This process takes a long time and consumes significant I/O and CPU resources.
Incremental compaction: Merges only the modified macroblocks, reusing unchanged macroblocks.
Incremental compaction significantly reduces the workload and is currently the default compaction algorithm in OceanBase AI Database.
Progressive compaction: Performs a full compaction on a portion of data each time, rewriting the entire dataset after several rounds.
Based on the operation granularity, major compactions can be classified as follows:
Tenant-level compaction: Freezes all tables within a tenant at a specified frozen SCN, forcing all tables in the tenant to merge to the same version.
Tenant-level compaction can be triggered in the following ways:
Automatic trigger: A major compaction is automatically triggered when the number of minor freezes reaches a specified threshold.
Scheduled trigger: You can specify the trigger time using the tenant-level parameter
major_freeze_duty_time, so that the compaction runs regularly every day.Manual trigger: You can execute the
ALTER SYSTEM MAJOR FREEZEstatement to manually trigger a major compaction.
Table-level compaction: Initiates a partition-level major compaction for all partitions of a specified table under a tenant. It does not guarantee that all partitions will merge to the same version.
Table-level compaction can be manually triggered by executing the
ALTER SYSTEM MAJOR FREEZE TABLE_ID = table_idstatement.Partition-level compaction: Performs a major compaction on all replicas of a specified partition based on a unified snapshot version, generating a new major SSTable.
Partition-level compaction can be triggered in the following ways:
Adaptive trigger: Automatically schedules a medium compaction for hot partitions based on various adaptive compaction strategies and buffer table strategies.
Manual trigger: You can execute the
ALTER SYSTEM MAJOR FREEZE TABLET_ID = tablet_idstatement to manually trigger a major compaction.
Major compaction status
You can view the status of major compactions through the status column in the DBA_OB_ZONE_MAJOR_COMPACTION view.
The main statuses are as follows:
IDLE: No major compaction is in progress.COMPACTING: A major compaction is in progress.VERIFYING: Verifying checksums.
Compression algorithms for major compaction
OceanBase AI Database does not flush small amounts of data to disk in real time. Instead, it performs major compactions to flush data to disk in bulk. Therefore, data can be compressed before being written to disk, which improves disk space utilization. You can choose the compression algorithm and method based on your business needs. For example, you can select a method with a high compression ratio but higher CPU consumption, or a standard compression method.
You can configure the compression method using the default_compress_func parameter. The default value is zstd_1.3.8. Other possible values are none, lz4_1.0, snappy_1.0, and zstd_1.0.
Note
A higher compression ratio saves more disk space but comes at a performance cost. For example, ZSTD typically saves more space than LZ4, but it takes longer for major compactions and requires a larger RT for I/O queries.
If you want to select a specific compression algorithm for an individual table, you can specify the algorithm when creating the table.
For the syntax on creating a table with a specified compression algorithm, see CREATE TABLE.
References
Major compaction