This topic describes the concepts, isolation methods, and application scenarios of resource isolation.
Control groups (cgroups) are a mechanism provided by the Linux kernel that allows a series of system tasks and their subtasks to be grouped (or separated) into different resource-level groups based on specific behaviors. This provides a unified framework for system resource management.
In the current version of OceanBase AI Database, tenant worker threads and most background threads are tenant-specific, while network threads are shared. Therefore, administrators can configure the cgroup directory and then use the DBMS_RESOURCE_MANAGER system package to control the CPU and IOPS usage of tenants.
DBMS_RESOURCE_MANAGER system package
OceanBase AI Database manages the allocation of resources in the database through the PL DBMS_RESOURCE_MANAGER system package to implement resource isolation. The DBMS_RESOURCE_MANAGER system package is mainly used to maintain the following elements:
Resource groups: A group of sessions that are combined based on resource requirements. The system allocates resources to a resource group rather than to individual sessions.
Resource management plans: Containers for resource management plan contents, which specify how resources are allocated to resource groups. You can control resource allocation by activating a specific resource management plan. One resource management plan can contain multiple plan contents. However, a resource management plan must not contain two identical plan contents.
Resource management plan contents: Used to associate a resource group with a resource management plan and specify how resources are allocated to the resource group.
For more information about the DBMS_RESOURCE_MANAGER system package, see Overview of DBMS_RESOURCE_MANAGER.
Resource isolation methods
When you configure resource isolation within a tenant by using the DBMS_RESOURCE_MANAGER system package, the isolation can be categorized based on the granularity of resource allocation as follows:
User-level resource isolation
User-level resource isolation is achieved by specifying the mapping between users and resource groups, so that all SQL statements executed by a user use resources allocated to the corresponding group.
SQL-level resource isolation
SQL-level resource isolation provides a finer granularity than user-level isolation. It is implemented by binding SQL statements that meet certain conditions to specified resource groups for execution. This approach is typically suitable in scenarios where multiple accounts exist in the business. When processing an order for one account, a transaction is started and a batch of SQL statements related to that account are executed (usually by specifying the account value in the WHERE clause). An account may consist of large accounts (with larger data volumes) and small accounts (with smaller data volumes). To prevent large accounts from consuming all CPU resources and preventing small accounts' orders from being processed, SQL statements for different orders can be bound to different resource groups. After binding, the SQL statements for different orders will use the resources of their respective resource groups.
Function-level resource isolation
Function-level resource isolation isolates the resources used by each task by specifying the mapping relationship between background tasks and resource groups. Function-level resource isolation is used to control the CPU and IOPS usage of background tasks.
Currently, it supports controlling the resource usage of the following types of background tasks corresponding to DAG threads:
- COMPACTION_HIGH: Mini Merge and DDL KV Merge tasks
- HA_HIGH: High-priority, high-reliability tasks such as replication, Rebuild, and recovery
- COMPACTION_MID: Minor Merge tasks
- HA_MID: Medium-priority, high-reliability tasks, such as migration tasks
- COMPACTION_LOW: Major Merge tasks
- HA_LOW: Low-priority, high-reliability tasks such as backup and backup cleanup
- DDL: Tasks for scenarios such as unique index verification and data replenishment after column deletion
- DDL_HIGH: DDL MemTable minor compaction tasks
- CLOG_LOW: CDC read tasks
- CLOG_MID: Tasks such as Follower replica reading from shared storage and log archiving reading from local disks
- CLOG_HIGH: Tasks such as writing clogs to the local disk, uploading clog files to shared storage, and reading from the local disk for follower replicas.
- OPT_STATS: Statistical information collection tasks.
- GC_MACRO_BLOCK: Macroblock recycling and reuse tasks.
- SQL_AUDIT: Storing SQL audit logs to disk.
- IMPORT: Import tasks.
- MVIEW: Materialized view background tasks.
- REPLAY_HIGH: Log replay.
- VECTOR_HIGH: Loading vector index memory data, loading/cleaning IVF caches (supported starting from V4.6.2.1).
- VECTOR_MID: Freezing and merging incremental segments of vector indexes (supported starting from V4.6.2.1).
- VECTOR_LOW: Partition-level rebuilding of vector indexes (supported starting from V4.6.2.1).