For boolean parameters, 0 indicates False and 1 indicates True.
Cluster configuration
Metadata configuration
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| init_log_level | Startup Log Level | ALL.*:INFO;SHARE.SCHEMA:INFO | No | The log level during startup, primarily used to output more detailed logs. It will be adjusted to the configuration of log_level after startup. |
| log_level | Log Level After Startup | ALL.*:INFO;SHARE.SCHEMA:WARN | Yes | None |
| max_log_file_count | Maximum Number of Log Files | 0 | No | The value range is greater than or equal to 0. A value of 0 indicates no log limit. To reduce the number of log files, you must restart obcdc. |
| enable_log_limit | Whether to enable log throttling | 1 | Yes | 1: Enable 0: Disable |
| rootserver_list | Specify the server information where the root service resides during startup. | |(English half-width) |
Yes | Format:server_ip:server_rpc_port:server_sql_port. |
| cluster_url | Cluster URL specified during startup | |(English half-width) |
No | When OceanBase has a cluster URL, obcdc is used to obtain rootserver information and supports high availability for the server. |
| cluster_user | Username in the sys tenant | Empty (user-specified, required) | No | You must configure a user in the sys tenant with read permission on internal tables (OceanBase databases). |
| cluster_password | Password of the above user | Empty (user-specified, required) | No | None |
| tenant_endpoint | List of Machines Where Tenant Resides | | (English half-width) |
No | Format:host:sql_portUse | Separated by . |
| tenant_user | Tenant username | | (English half-width) |
No | Format:user_name@tenant_nameoruser_name@tenant_name#cluster. |
| tenant_password | Password corresponding to the tenant username above | | (English half-width) |
No | None |
| config_fpath | Configuration File Dump Path | etc/liboblog.conf | No | All configuration information of obcdc is dumped to this file. |
| timezone | Time zone of obcdc | +8:00 | No | The time zone that affects data formatting output must be consistent with the data source configuration. The default is UTC+8. |
| timezone_info_path | Specifies the directory where the timezone metadata files are located. | etc/timezone_info.conf | Yes | meta_data_refresh_modeSet todata_dictWhen you do so, you need to extract the files from the RPM package.timezone_info.confFiles are placed in this directory. |
| skip_ob_version_compat_check | Whether to skip the OceanBase version compatibility check | 0 | No | obcdc in OceanBase Database V3.x supports only synchronizing data from OceanBase Database V2.x or V3.x, and the version number of obcdc must be greater than that of OceanBase Database (when comparing, consider both the major and minor version numbers). |
| cluster_version_refresh_interval_sec | Query the minimum version number of OBServer nodes in an OceanBase cluster | 600 | No | Specifies the interval for refreshing the OceanBase cluster version, in seconds. |
| log_router_background_refresh_interval_sec | Log Stream Metadata Refresh Frequency | 10 | Yes | The interval at which the backend refreshes the server information corresponding to the log stream, in seconds. |
Working mode related
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| working_mode | Operating Mode | storage (commercial edition) | No | Try to control the transaction size for your business. If large transactions are present in your business scenario, it is recommended to use the persistent mode. In persistent mode, it is recommended to deploy obcdc on an SSD disk. |
| store_service_path | Data storage path in obcdc persistent mode | ./storage | No | Effective only in persistent mode. Relative path: A relative path stored under the directory where the process calling obcdc resides. Absolute path: Stored under a specified absolute path, for example, store_service_path=/data/1. |
Synchronization granularity
obcdc supports synchronizing data in multiple dimensions. A single obcdc instance can synchronize a cluster, one or multiple tenants, several databases, or several tables.
Synchronization Granularity |
Single OBDC |
Multiple obcdc instances |
|---|---|---|
| cluster | Supported (Single OBDC cannot synchronize multiple clusters.) | Supported |
| tenant | Supported | Supported |
| database | Not supported | Not supported |
| table | Not supported | Not supported. (See the multi-instance parameters below.) |
If a tenant has very high traffic and it is difficult for a single-instance obcdc to achieve real-time synchronization of the entire cluster, you can use multiple obcdc instances to synchronize the same cluster to share the load. Each instance synchronizes data from different tenants.
Allowlist and blocklist related configuration
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|
|---|---|---|---|---|---|
| enable_white_black_list | Controls whether to enable allowlist and blocklist for obcdc. | 1 | No | Enabled by default. The blocklist/allowlist configuration takes effect. After it is disabled, data from the entire tenant will be synchronized. The scope of tenants to be synchronized is limited to those configured in the allowlist. | |
| tb_white_list | Synchronized Table Allowlist | *.*.* | No | Format: tenant name.database name.table name (for example, mytenant1.db1.tb1\ | mytenant2.db2.*). |
| tb_black_list | Blacklist of Synchronized Tables | | |
No | The format is the same as above. | |
| tablegroup_white_list | Synchronized Tablegroups Allowlist | *.* | No | Format: tenant name.tablegroup name, for example, (mytenant1.*\ | or_tenant*.tg*). |
| tablegroup_black_list | Synchronized Tablegroups in Blocklist | | |
No | The format is the same as above. | |
| enable_filter_sys_tenant | Whether to filter data synchronized from the sys tenant | 0 | No | This mode is enabled when the sys tenant is abnormal but the user tenant is normal. |
Note
obcdc provides data filtering capabilities based on table and tablegroup allowlists and blocklists:
- Only tables/tablegroups that are in the allowlist and not in the blocklist are synchronized. The default allowlist includes all tables/tablegroups under all tenants, and the default blocklist is empty. It is recommended to use the allowlist in production environments to synchronize data for specified tenants, rather than using the default values (which will pull redundant data).
- The configuration uses fnmatch for regular expression matching. The default matching rule is case-insensitive. If enable_oracle_mode_match_case_sensitive=1 is configured, name matching is case-sensitive for tenants in ORACLE-compatible mode.
Multi-instance related configuration
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| instance_num | Total obcdc Processes Sharing Traffic | 1 | No | None |
| instance_index | Index of the current obcdc in the total count (can be understood as the hash bucket ID) | 0 | No | The value range is [0, instance_num). Each OBDC only processeshash(tenant_id, ls_id) % instance_num = instance_idxof the log stream data. |
| enable_global_unique_index_belong_to_multi_instance | Whether to allow synchronizing globally unique index tables in multi-instance scenarios | 0 | Yes | A value of 0 indicates that synchronizing globally unique index tables across multiple instances is not allowed. |
Monitoring related configuration
Throttling
To prevent Out-of-Memory (OOM) issues caused by data accumulation in memory under abnormal conditions, obcdc provides throttling capabilities. Throttling is triggered based on the internal state of obcdc, pausing the pulling of new data until the obcdc state recovers. The throttling function is controlled by the following parameters. Adjust the corresponding parameters as needed to modify the throttling level.
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| memory_limit | Memory Limit for obcdc When Flow Control Is Triggered | 20G |
Yes | One of the flow control reference parameters, which aims to keep the memory usage of obcdc withinmemory_limitNearby" (which OBDC tries to control) does not mean that OBDC uses at mostmemory_limitThe minimum memory size is 2 GB. |
| memory_usage_warn_threshold | Sets the memory usage threshold for triggering log pull throttling (in percentage).memory_limit(compared to) |
85 | Yes | None |
| system_memory_avail_percentage_lower_bound | Minimum Available System Memory Threshold (Percentage) | 10 | Yes | Value range: [10, 80]. |
| part_trans_task_active_count_upper_bound | Maximum Active Partition Transactions | 200000 | Yes | The number of active partition transaction structures in memory (allocated and not reclaimed). |
| storager_task_count_upper_bound | Throttling threshold for the number of tasks to be persisted by the storager module | 1000 | Yes | The minimum value is 1. |
| storager_mem_percentage | Controls the multiple by which the persistence module occupies relative flow control memory | 1 | Yes | A value of 0 indicates automatic adjustment. |
| part_trans_task_reusable_count_upper_bound | Maximum Number of Partition Transactions to be Reused | 10240 | Yes | The number of tasks that are pending output to the user queue, pending user output, pending user return, and not reclaimed by the garbage collector. |
| ready_to_seq_task_upper_bound | Maximum Number of Pending Tasks | 2000 | Yes | The maximum number of pending-sequence tasks is the sum of the number of tasks pending parsing, formatting, and persistence. |
| pause_fetcher | Force Log Pull Suspension | 0 | Yes | A value of 0 indicates that the result of the flow control calculation logic is ignored, and log pulling is forcibly paused. |
Throttling reference conditions:
- Condition 1: The number of active partitions exceeds the upper threshold (part_trans_task_active_count_upper_bound) || The memory held by obcdc exceeds the upper threshold (memory_limit) || The system available memory is lower than the lower threshold (system_memory_avail_percentage_lower_bound).
- Condition 2: The number of partition transaction structures pending reuse exceeds the upper threshold (part_trans_task_reusable_count_upper_bound) || The number of tasks ready for sequencing exceeds the upper threshold (ready_to_seq_task_upper_bound).
- Condition 3 (since 3.x): The number of tasks pending persistence exceeds the threshold (storager_task_count_upper_bound) && The memory held by obcdc exceeds the flow control trigger memory (memory_limit) * The ratio of the memory occupied by the persistence module to the flow control memory (storager_mem_percentage).
The flow control is triggered when: ((Condition 1 && Condition 2)|| Condition 3) || (pause-fetcher != 0).
Monitoring information output
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| enable_dump_pending_trans_info | Controls whether to periodically collect statistics on and print the status of distributed transactions held by obcdc. | 0 | Yes | A value of 0 indicates that the status of distributed transactions held by OBDCD is not periodically collected and printed. If configured as1Then it will be printed every 10 seconds. |
| print_fetcher_slowest_ls_num | Print the number of partitions with the slowest log pulling | 10 | Yes | None |
| print_rpc_handle_info | Controls whether to print RPC-related information. | 0 | Yes | A value of 0 indicates that RPC-related information is not printed. |
| print_stream_dispatch_info | Controls whether to print log stream pull information | 0 | Yes | A value of 0 indicates not to print log stream pull information. |
| print_ls_heartbeat_info | Controls whether to print partition heartbeat information. | 0 | Yes | A value of 0 indicates not to print partition heartbeat information. |
| print_ls_serve_info | Controls whether to print partition service information (partition status, number of partition transactions). | 0 | Yes | A value of 0 indicates that partition service information (partition status, number of partition transactions) is not printed. |
| print_participant_not_serve_info | Controls whether to print partition transaction information for all distributed transactions that are not serving (do not output data). | 0 | Yes | A value of 0 indicates not to print partition transaction information for all distributed transactions that are not serving (do not output data). |
| print_ls_server_list_update_info | Controls whether to print detailed information about the server list when updating it. | 0 | Yes | A value of 0 indicates that detailed information about the server list is printed when the partition server list is not updated. |
| enable_formatter_print_log | Controls whether to allow the formatter to output logs for partially unformatted data. | 0 | Yes | A value of 0 indicates that the formatter is not allowed to output logs with partially unformatted data. |
Feature control related configurations
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| enable_output_virtual_generated_column | Controls whether to output virtual generated columns | 0 | No | Withenable_output_by_table_defIn conjunction with
NoteThis parameter is available starting with V4.2.5. That is, when OceanBase Database is V4.2.5 or later in the V4.2.x series, CDC supports outputting virtual generated columns. |
Metadata management
obcdc version 4.1 supports obtaining metadata (such as schema/log stream information) through the data dictionary, eliminating the need to send RPCs to OBServer nodes to pull logs.
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| meta_data_refresh_mode | Specifies the method for obtaining metadata such as schemas and log streams. | online | No |
NoticeWhen synchronizing data from OceanBase Database V4.2 and later, obcdc ignores user configurations and forcibly uses |
Tenant management
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| skip_rename_tenant_ddl | Whether to ignore DDL for renaming tenants | 0 | Yes | If a renamed tenant is not within the OBDC synchronization allowlist, no impact occurs. Otherwise, an error is reported and the process exits normally (because the rename operation violates the allowlist restrictions). In this case, it is recommended to modify the allowlist to support both the tenant before and after the rename. When this parameter is set to true, the DDL for renaming the tenant is skipped, and data synchronization continues. |
| enable_oracle_mode_match_case_sensitive | Case sensitivity in blocklist/allowlist matching for OceanBase-Oracle mode tenants | 0 | No | None |
Fetcher module related configurations
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| ls_count_upper_limit | Number of Partition Progresses | 2000000 | No | Up to 200 million partitions are supported. |
| check_switch_server_interval_min | Periodically Check Whether Partition Switchover for Log Pull is Needed | 30 | Yes | None |
SQL query related configurations
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| access_systable_helper_thread_num | Maximum Number of Connections in the Connection Pool for Executing SQL | 64 | No | Value range: [48, 1024]. |
| rs_sql_connect_timeout_sec | SQL Connection Timeout for Root Service | 40 | Yes | None |
| rs_sql_query_timeout_sec | SQL Request Timeout Period for Root Service | 30 | Yes | None |
| tenant_sql_connect_timeout_sec | SQL Connection Timeout Period for Tenant | 40 | Yes | None |
| tenant_sql_query_timeout_sec | SQL Request Timeout Period for Tenant | 30 | Yes | None |
| sql_server_change_interval_sec | Cycle for switching execution servers of SQL requests | 60 | Yes | None |
Filter data based on cluster_id
obcdc provides data filtering capabilities based on a cluster_id blacklist: OceanBase defines a valid cluster_id range, but to prevent synchronization of unwanted data, users can specify a cluster_id when writing data. To configure the cluster_id for writing data at the source, refer to the ob_org_cluster_id description.
- Do not synchronize data whose cluster_id is outside the valid range specified by OceanBase and OMS ([1, 4294901759] U [4294901760, 4294967295]).
- Do not synchronize data whose cluster_id is in the cluster_id_black_list.
- Each cluster_id in the cluster_id_black_list must be within the range of cluster_id_value_min and cluster_id_value_max.
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| cluster_id_black_list | Synchronized Cluster ID Blocklist | | |
No | Separate multiple cluster IDs with half-width characters |, where each cluster_id value is between cluster_id_value_min and cluster_id_value_max. |
| cluster_id_black_value_min | Minimum Cluster ID in Synchronization Blocklist | 2147473648 | No | None |
| cluster_id_black_value_max | Maximum Synchronized Cluster IDs in Blocklist | 2147483647 | No | None |
Server-level blacklist
obcdc initiates SQL and RPC requests to OBServer nodes. We provide configuration items to block specific SQL execution machines and RPC execution machines (effective for all requests).
Additionally, since obcdc pulls logs at the partition level, it has a built-in partition-level blacklist for log pulling. Machines that time out or report errors when pulling partition logs are added to this partition log pulling machine blacklist. The process of adding a machine to the blacklist and removing it from the blacklist is automatic.
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| server_blacklist | Configure the RPC request machine blacklist. obcdc does not send RPC requests to machines on this list. | | |
Yes | The format issvr_ip:rpc_portMultiple OBServer nodes are connected by using the | Separated by;`. Machines in this list are not used to pull logs. You can manually add an OBServer node to this list if it cannot provide log service normally (for example, due to network partitioning or excessive latency). |
| sql_server_blacklist | Configure a blocklist of machines for SQL requests. obcdc will not send SQL requests to machines in this blocklist. | | |
Yes | The format issvr_ip:rpc_portMultiple OBServer nodes are connected using the | Separated by . |
Partition-level log pull server blocklist regulation (usually unchanged)
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| blacklist_survival_time_sec | Waiting Time of Machines in the Blocklist | 30 | No | None |
| blacklist_survival_time_upper_limit_min | Maximum Time for Machine in Blocklist | 4 | Yes | None |
| blacklist_survival_time_penalty_period_min | Threshold for Doubling Blocklist Duration When a Machine Repeatedly Adds to the Blocklist | 1 | Yes | None |
| blacklist_history_overdue_time_min | How long will a machine remain in the blocklist history before being deleted? | 30 | Yes | Minimum value: 10 |
| blacklist_history_clear_interval_min | Cycle for Cleaning Machine Information from the Blocklist | 20 | Yes | Minimum value: 10 |
Server cache information related configuration
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
|---|---|---|---|
| all_server_cache_update_interval_sec | Cache update cycle for server list information | 5 | Yes |
| all_zone_cache_update_interval_sec | Cache update cycle for zone information | 5 | Yes |
Log pull related configuration
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| idle_pool_thread_num | Number of Threads for Processing Pending Partition Log Pull Tasks | 4 | No | Value range: [1, 32]. |
| dead_pool_thread_num | Number of Threads for Processing Log Pulling Tasks from Recycled Partitions | 1 | No | Value range: [1, 32]. |
| stream_worker_thread_num | Number of Threads for Distributing Partition Log Pull Tasks | 8 | No | The value range is [1,64]. |
| start_lsn_locator_thread_num | Number of Threads for Locating the Partition Log Start | 4 | No | Value range: [1, 32]. |
| lob_data_merger_thread_num | Number of Lobs Processing Threads | 2 | No | The value range is [1, +∞). |
| start_lsn_locator_locate_count | Number of Requests for Locating the Start of Logs Completed | 1 | No | The minimum value is 1. To locate the starting log ID of a partition, you need to ensure that a specified number of partition replicas return the same starting log ID. |
| skip_start_lsn_locator_result_consistent_check | Skip consistency check for start log IDs | No | No | None |
| svr_stream_cached_count | Initial Pool Size of Machine Log Stream Objects | 16 | No | The minimum value is 1. |
| fetch_stream_cached_count | Initial Size of Partition Log Stream Object Pool | 16 | No | The minimum value is 1. |
| start_lsn_locator_rpc_timeout_sec | Timeout Duration for RPC to Locate the Start Log ID of a Partition | 60 | Yes | The minimum value is 1. |
| start_lsn_locator_batch_count | RPC Batch Size for Locating the Start Log ID of a Partition | 2000 | Yes | The minimum value is 1. |
| heartbeat_interval_sec | Partition Heartbeat RPC Cycle | 1 | Yes | The minimum value is 1. |
| fetch_log_rpc_timeout_sec | Pull Partition Log RPC Timeout Period | 15 | Yes | The minimum value is 1. |
| progress_limit_sec_for_dml | Maximum DML Data Pull Log Progress | 300 | Yes | The minimum value is 1. To ensure that the progress of all partitions is roughly consistent (preventing some partitions from pulling logs too quickly), the upper limit for logs provided by the server is set to the minimum progress of all partitions plus progress_limit_sec_for_dml. |
| ls_fetch_progress_update_timeout_sec | Threshold for RPC Timeout in Partition Log Pull | 3 | Yes | The minimum value is 1. |
| ls_fetch_progress_update_timeout_sec_for_lagged_replica | Timeout Threshold for Pulling Backlogs on Lagging Replicas | 15 | Yes | The minimum value is 1. |
| enable_continue_use_cache_server_list | Server information on whether to use cache | 0 | No | This mode is enabled when the sys tenant is abnormal but the user tenant is normal. |
| fetching_log_mode | Used to specify the log pull mode. | integrated | No |
Noteobcdc does not support pulling discontinuous archived logs. |
Log pull priority related
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| region | Configure the Region for Obcdc to Preferentially Pull Logs | default_region | Yes | Regions need to be retrieved from internal tables of OceanBase Database. obcdc will preferentially pull logs from OBServer nodes within the configured regions. |
RPC module configuration (not recommended for modification)
Parameter |
Meaning |
Default Value |
Value range |
Dynamic Change Allowed |
|---|---|---|---|---|
| io_thread_num | Number of RPC IO Threads | 4 | [1, +∞) | No |
| rpc_result_cached_count | Initial Pool Size for Log Pull RPC Result Objects | 16 | [1, +∞) | No |
| rpc_process_handler_time_upper_limit_msec | Maximum RPC Processing Time | 200 | [1, +∞) | Yes |
| rpc_result_count_per_rpc_upper_limit | The maximum number of outstanding log pull RPCs allowed in a single log stream. Once this limit is reached, log pull RPCs will be paused. | 16 | [1, +∞) | Yes |
Secure communication related
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| ssl_client_authentication | Whether to enable secure communication | 0 | No | The default value is 0, which indicates that secure communication is not enabled. |
| ssl_external_kms_info | Secure Communication Method | file | No | For external use, only the file mode is supported. The key files are placed in the wallet directory under the obcdc startup directory (please create your own wallet directory), which contains the following key files: ca.pem/client-cert.pem/client-key.pem. |
Sequencer module related configuration
Parameter |
Meaning |
Default Value |
Value range |
Dynamic Change Allowed |
|---|---|---|---|---|
| sequencer_thread_num | Distributed Transaction Scheduling Thread Pool Size | 5 | [1, +∞) | No |
| sequencer_queue_length | Partition Transaction Queue Pending Order | 102400 | [1, +∞) | No |
REDO read control module
Parameter |
Meaning |
Default Value |
Value range |
Dynamic Change Allowed |
|---|---|---|---|---|
| redo_dispatched_memory_limit_exceed_ratio | When enable_output_trans_order_by_sql_operation=1, it specifies the size limit for redos to be processed and distributed (that is, the maximum size of redos allowed to be processed in memory is redo_dispatched_memory_limit_exceed_ratio * redodispatcher_memory_limit). | 2 | [1, +∞) | Yes |
| redo_dispatcher_memory_limit | Limit the Redo Size for Control Distribution Processing | 512M | [128M, +∞) | Yes |
| extra_redo_dispatch_memory_size | A REDO balance allocated to participants with data skew to prevent OBDC from not outputting data due to deadlock. | 4M | [0M, 512M] | Yes |
Row data sorting module
Parameter |
Meaning |
Default Value |
Value range |
Dynamic Change Allowed |
|---|---|---|---|---|
| msg_sorter_task_count_upper_limit | Maximum Number of Row Data Sorting Module Tasks per Transaction | 200000 | [1, +∞) | No |
| msg_sorter_thread_num | Number of Threads for Row Data Sorting Module in Transactions | 1 | [1, 32] | No |
Persistence mode effective configurations
Storager module related configuration
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
|---|---|---|---|
| storager_thread_num | Thread Pool Size of the Persistence Module | 10 | No |
| storager_queue_length | Number of Pending Persistence Tasks | 102400 | No |
| rocksdb_flush_interval | Period for Active FLUSH of ROCKSDB | 10 Minutes by Default | Yes |
| rocksdb_compact_interval | Period for Active COMPACT ROCKSDB | 6 hours by default | Yes |
Data Processor module related configuration
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
|---|---|---|---|
| reader_thread_num | Thread Pool Size for Reading Data from the Persistence Module | 10 | No |
| reader_queue_length | Number of Tasks Awaiting Deserialization | 102400 | No |
REDO batch persistence related configuration
Parameter |
Meaning |
Default Value |
Value range |
Dynamic Change Allowed |
|---|---|---|---|---|
| batch_buf_count | Number of Memory Buffers for Redo Persistence | 10 | [5, +∞) | No |
| batch_buf_size | Aggregate Buffer Size for Redo Batch Persistence | 20MB | [2MB, +∞) | No |
Parser module configuration
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| dml_parser_thread_num | Thread Pool Size for Parsing DML Statements | 5 | No | The minimum value is 1. |
| ddl_parser_thread_num | Thread Pool Size for Parsing DDL Statements | 1 | No | The minimum value is 1. |
| skip_reversed_schema_version | Ignore schema version rollback exception | 0 | No | The DDL processing is generally a process of monotonically increasing the schema version. If the schema version bounces back, it indicates an exception. This feature provides configuration to skip such exceptions. |
Formatter module configuration
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| formatter_thread_num | Data Formatting Thread Pool Size | 10 | No | The minimum value is 1. |
| skip_dirty_data | Skip outliers | 0 | Yes | Current exceptions:
|
Output Binlog Record configuration
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| enable_output_trans_order_by_sql_operation | Whether to output row data within a transaction in the order of SQL execution | 0 | No | The internal data modification order in OceanBase Database may not exactly match the SQL execution order specified by the client. |
| sort_trans_participants | Sort by Distributed Transaction Participant | 1 | No | It refers to sorting by the partitioning key. When combined with enable_output_trans_order_by_sql_operation, this can result in a stable output order of statements. |
| enable_output_hidden_primary_key | Whether to output the hidden primary key for tables without a primary key | 0 | No | For tables without a primary key, you can set enable_output_hidden_primary_key=1 to output the hidden primary key. |
| enable_convert_timestamp_to_unix_timestamp | Whether to convert the timestamp to a UNIX integer time | 0 | No | None |
| enable_output_invisible_column | Whether to output hidden columns | 0 | No | None |
| enable_output_by_table_def | Controls whether to output columns in the user-defined column order. | 0 | No | None |
| output_heartbeat_interval_msec | Interval for Outputting Safety Point Information | 1000 | Yes | None |
Resource recycling configuration (Not recommended to modify)
Parameter |
Meaning |
Default Value |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|
| resource_collector_thread_num | Resource Recycling Thread Pool Size | 10 | No | The minimum value is 1. |
| resource_collector_thread_num_for_br | Thread Pool Size for Binlog Record Recycling | 7 | No | It must be greater than resource_collector_thread_num. |
Notice
Modifications to configurations not described in this document are not recommended.
Emergency configuration
Emergency configurations to ensure the normal operation of obcdc in case of OceanBase Database exceptions:
Parameter |
Meaning |
Default Value |
Value range |
Dynamic Change Allowed |
Remarks |
|---|---|---|---|---|---|
| lsn_black_list | Control to Ignore Logs with Specific LSN Ranges | | |
Not involved | No | For emergency use only. |
| skip_delete_tablet_op | Controls whether to ignore delete tablet operations in obcdc. | 0 | {0, 1} | Yes | For emergency use, some direct load/offline DDL operations may cause errors in the obcdc parser module.ENTRY_NOT_EXISTThis configuration is required to skip errors during emergency configuration. It is available starting from V4.2.1 BP2. |
| skip_empty_outrow_lob_update | Controls whether obcdc ignores null outrow LOB exceptions. | 0 | {0, 1} | Yes | For emergency use only. |