This topic describes the new features and changes of OceanBase Database Community Edition V3.1.4 based on the previous version.
Overview
OceanBase Database Community Edition V3.1.4 is optimized based on the feedback of community users to improve the ease of use and stability of the kernel.
Kernel improvement: Data files can be dynamically allocated as needed, and statistics of monitoring metrics can be collected and displayed in histograms.
Stability improvement: The multi-model OceanBase Database-KV (OBKV) can clear expired historical data and supports larger transactions of a single table.
Easier to use: You can delete one plan at a time from the plan cache. The display of logs and error codes is optimized. ob-operator supports the deployment of ODP. The latest open source test framework is released.
New features
Dynamic allocation of data files as needed
OceanBase Database is a distributed disk-resident database management system. It organizes and manages data in a logical system architecture that consists of macroblocks and microblocks, and uses SSBlock files for physical data storage. At present, OceanBase Database manages the disk space by fully allocating the maximum space for SSBlock files in advance. This method ensures the availability of disk resources during the use of OceanBase Database, avoids data persistence failures due to insufficient disk space, and reduces the runtime risks of the system.
In addition, this version supports the dynamic allocation of data files as needed. You need to specify only the datafile_next parameter, which specifies the automatic increment step of disk occupation, and the datafile_maxsize parameter, which specifies the maximum disk space that can be automatically allocated for SSBlock files.
OceanBase Database uses scheduled tasks for disk space adjustment based on the following strategies:
The system calculates and measures disk usage in the background by launching scheduled tasks based on the specified value of disk space for the SSBlock files. When disk usage reaches the specified percentage, such as 95%, automatic scaling of disk space is triggered.
When the allocated disk space is insufficient, the system automatically increase the disk space based on your configurations.
The space for data files on a single OBServer node cannot be recycled. If you want to reduce and release the occupied disk space, you can scale in the cluster by removing servers.
Display of monitoring statistics in histograms
This new version displays collected monitoring statistics in histograms. Based on the Prometheus histogram metrics, OceanBase Database defines and generates internal statistics tables to record statistics, such as the statistics at the 99th and 95th percentiles, which are then exported to Prometheus by the open source obagent component for display. This feature improves the product operation easiness.
Notice
If this feature is enabled, the write performance is slightly reduced.
The histogram statistics of OceanBase Database provide the following features:
Supports the QUERY_RESPONSE_TIME view, which is defined in the INFORMATION_SCHEMA database.
Supports query_response_time_stats, the system variable that specifies whether to enable statistics.
Supports query_response_time_range_base, the system variable that specifies the time interval of statistics collection.
Supports query_response_time_flush, the system variable that specifies whether to refresh statistics and reread data.
Supports OBAgent to read the statistics tables and generates histograms in Prometheus.
Cleanup of expired historical data in the multi-model OBKV database
The multi-model OBKV database can clear expired historical data.
OceanBase Database maintains expired data on the server and supports data recycling based on the TIMESTAMP and MAXVERSION attributes. You can specify the expiration time and the maximum number of versions of table data in the COMMENT column of the column family table. OceanBase Database runs periodic background tasks to scan transactions and clean up the expired data by partition at the Partition Service layer.
Core enhancements:
The __all_kv_ttl_task and __all_kv_ttl_task_history tenant-level system tables are provided to record the status and operation history of the internal periodic TTL-based tasks. To maintain the number of records in the __all_kv_ttl_task_history table, garbage collection (GC) is performed on this table based on the GC strategies that are similar to those of other history tables.
The DBA_OB_KV_TTL_TASKS and DBA_OB_KV_TTL_TASK_HISTORY views are provided for you to view the ongoing and historical TTL-based tasks in the tenant.
The __all_virtual_kv_ttl_task and __all_virtual_kv_ttl_task_history virtual tables, and the corresponding CDB_OB_KV_TTL_TASKS and CDB_OB_KV_TTL_TASK_HISTORY views are provided for you to log onto the sys tenant and view the ongoing and historical TTL-based tasks in all business tenants.
The kv_ttl_duty_duration parameter is provided to specify the execution period of the background TTL-based tasks. If a task is not completed at the end of the specified execution period, it is suspended and then resumed when the specified execution period starts again. You can specify the execution period of the background TTL-based tasks based on the characteristics of your business.
The enable_kv_ttl and kv_ttl_history_recycle_interval parameters are provided. The enable_kv_ttl parameter specifies whether to enable the background TTL-based tasks. The kv_ttl_history_recycle_interval parameter specifies the period for which the history of the TTL-based tasks is retained.
Memory-usage-based throttling is provided. OceanBase Database dynamically adjusts the progress of the background TTL-based tasks based on memory usage of MemStore.
Transaction throttling is provided. OceanBase Database limits the number of table records that are queried and deleted each time in a background TTL-based task, commits transactions in batches to avoid large transactions, and supports transaction rollback in case of lock contention.
Removal of cached SQL execution plans one at a time
When an unreasonable SQL execution plan is diagnosed, the DBA can use the SQL ID to remove only this plan from the plan cache. This prevents system instability due to the purge of the entire plan cache. Usage notes:
The sys tenant can invalidate a specific or all SQL execution plans of a user tenant by running the following command:
ALTER SYSTEM FLUSH PLAN CACHE [ [SQL_identifier] [db_list] tenant_list ][global].A MySQL tenant can invalidate a specific or all SQL execution plans of the tenant by executing the following statement:
ALTER SYSTEM FLUSH PLAN CACHE [ [ SQL_identifier ] [db_list] ] [global];
Enhanced product operation easiness
ob-operator supports the deployment of ODP, which makes it easier to use a multi-node cluster and improves the accuracy of query routing in the cluster.
To be compatible with MySQL error codes, OceanBase Database of earlier versions converts the internal error codes to external error codes when the log is generated. As a result, the error codes displayed at the client are different from those recorded in the log, which adds extra workload to troubleshooting. To make the troubleshooting easier, this version prints the external error codes that are exported to the client and the corresponding internal error codes at the same time.
If the execution of the
update [table_name] set [column_name];statement fails, information of the faulty columns is indicated in the error messageData too long for column.The latest versions of open source OBClient and test-driving framework are provided to make code development and testing more accessible to developers in the community.
Behavioral changes
The lower_case_table_names variable cannot be modified.
The lower_case_table_names variable is a global read-only variable. You cannot modify the variable by executing the
set global lower_case_table_names;oralter tenant [tenant_name] set variables lower_case_table_names;statement.If the automatic scaling feature is enabled, the statistical value of the total_size column of the __all_virtual_disk_stat table specifies the maximum disk space for data storage.
Fixed issues
The datadir variable uses the content of the cluster parameter
data_dir. This system variable is read-only and cannot be modified.The internal limit on large transactions and the limit on the size of the log_id of transaction groups are removed, and the _max_trx_size parameter is provided to avoid the negative impact on the database due to ultra-large transactions.
A table name cannot contain more than 64 characters.
The
Invalid argumenterror is returned during the execution of a query with the/*+ LEADING */hint.The error code 4016 is returned when you create a table by using the
create table t select ~1;syntax.Multi-line comments for SQL statements are supported.
The definition of the virtual table corresponding to the
show full columnscommand is modified to improve the compatibility of OceanBase Database with MySQL Database.