lake_table_pruning_thread_count specifies the number of threads in the thread pool for parallel pruning of lake tables, providing thread resources for parallel pruning of Iceberg and Paimon tables.
Overview
When enable_lake_table_parallel_resolving is enabled, the system uses a tenant-level thread pool to perform parallel pruning tasks. The lake_table_pruning_thread_count parameter determines the number of threads in this thread pool. A larger number of threads results in higher concurrency for parallel pruning, shorter plan generation time, but also consumes more CPU and memory resources.
Attributes
Attribute |
Description |
|---|---|
| Parameter type | INT |
| Default Value | 4 |
| Value range | [1, 64] |
| Modifiable | Yes, you can use the following methods:ALTER SYSTEM SETStatement modification. |
| Whether to restart the OBServer node for the change to take effect | No, the setting takes effect immediately. |
Usage instructions
For tenants with ample resources and a large number of data files, you can appropriately increase the number of threads to shorten plan generation time. For tenants with limited resources or a high volume of concurrent queries, you can appropriately reduce the number of threads to prevent excessive resource consumption by thread pruning.
Configuration example
-- View Current Value
SHOW PARAMETERS LIKE 'lake_table_pruning_thread_count';
-- Adjust the number of threads to 8.
ALTER SYSTEM SET lake_table_pruning_thread_count = 8;
-- Adjust the number of threads to 16.
ALTER SYSTEM SET lake_table_pruning_thread_count = 16;