OceanBase Database Community Edition V4.1.0

V4.1.0_CE_BP4

Version information

• Release date: September 21, 2023
• Version: V4.1.0_CE_BP4

Overview

This version addressed several customer and internal testing issues to enhance product stability.

Bug fixes

• Fixed the issue where a size overflow problem might occur in a SQL statement if there were a large number of OR conditions or if a large number of OR conditions were generated after the SQL statement was rewritten.
• Fixed the issue where the index skip scan plan was used to query partitioned tables due to inaccurate statistics, resulting in memory bloat in the TableRescanCtx module.
• Fixed the issue where new backup tenants might encounter a request queue backlog problem during the switchover between primary and backup nodes.
• Fixed the issue where there was an inconsistency between the data obtained through vectorized table scan and non-vectorized table scan due to vectorized decoding exceptions.
• Fixed the issue where after creating a function-based index, if the result of the function-based index expression depended on system variables, Error 4377 might be reported when modifying the system variables and then performing update or delete operations. Added Error code 5500 to distinguish this type of issue.
• Fixed the issue where a disconnection problem might occur during the upgrade from OceanBase Database Community Edition V4.1.0 to V4.2.0. If the source version is earlier than V4.1.0 BP4, this issue can be avoided by first upgrading OceanBase Database and then upgrading OceanBase Database Proxy (ODP).
• Fixed the issue where the disk space calculation error after migration caused the unit to be migrated back and forth.
• Fixed the issue where OBKV failed to use local indexes for queries when partitioned tables were used.
• Fixed the issue of continuous reporting of Error 4013 and inability to successfully refresh the schema when the OBServer node was restarted. The cause of this issue was that the number of records in the __all_virtual_table_history table of a tenant in the cluster exceeded 4 million.
• Fixed the following issue: When performing a Partition Wise Join operation on two partitioned tables with slightly different partitioning methods, but with the same partitioning method after partition pruning, different results might be returned under different degrees of parallelism.
• Fixed the issue where MySqlConnector (a .NET driver) encountered an error when connecting to OceanBase Database and executing statements related to stored procedure.

V4.1.0_CE_BP3_HF1

Version information

• Release date: August 28, 2023
• Version: V4.1.0_CE_BP3_HF1
• RPM version: oceanbase-ce-4.1.0.2-103010012023082519

Overview

This version addressed several customer and internal testing issues to enhance product stability.

Bug fixes and improvements

• Fixed the issue where modifying tenant specifications caused a limitation of 100 connections for non-root users.
• Fixed the issue where concurrent access to the plan_cache_plan_stat table during the deletion of an execution plan could trigger a core dump.
• Added a new parameter to control the optional plan paths for the JOIN operator.

Considerations

OceanBase Database Community Edition can be directly upgraded from V4.1.0_CE_BP3_HF1 to V4.2.0_CE_GA but not to V4.2.0_CE_BETA.

V4.1.0_CE_BP3

Version information

• Release date: August 14, 2023
• Version: V4.1.0_CE_BP3
• RPM version: oceanbase-ce-4.1.0.2-103000072023081111

Overview

1. OBKV now supports aggregate functions (such as MIN, MAX, and COUNT), auto-increment columns, and row data verification during data insertion.
2. Two hidden parameters _ha_get_tablet_info_batch_count and _ha_rpc_timeout are added to control the number of tablets obtained per RPC call and the RPC timeout during operations such as migration replication, rebuild, and physical restore. These parameters are suitable for scenarios where the SATA disk I/O is slow.
3. The DBMS_RESOURCE_MANAGER system package is now available in the Community Edition.
4. This version addressed several customer and internal testing issues to enhance product stability.

Bug fixes

• Fixed the issue where after the LIMIT pushdown statement is rewritten, the execution result of a LIMIT clause containing the RAND function is incorrect.
• Fixed the issue where if an expression contains functions such as USERENV and SYSDATE, the query result is inaccurate due to an incorrect query range.
• Fixed the issue where in the Prepared Statement (PS) protocol, a zero datetime value of the DATETIME type is not fully compatible with MySQL when an OBServer node sends a packet to the client. As a result, the query result on the client of the Go driver is abnormal.
• Fixed the issue where when the GROUP BY clause is used in a join of two tables, the returned result set does not meet the expectations.
• Fixed the issue where in a scenario where the cgroup feature is enabled and sql_nio is disabled, a core dump may occur due to the use of the PS protocol.
• Fixed the issue where if the CPU resources remain exhausted for a period of time, an OBKV request may time out and thereby cause a core dump.
• Fixed the issue where after the character set of a tenant is changed by using the ALTER DATABASE statement, the character set displayed in the information_schema.SCHEMATA view is incorrect.
• Fixed the issue where when a large number of DDL operations are performed on a single table, the minor compaction threads may be stuck due to too many involved partitions or intensive writes, which results in an out of memory (OOM) error.
• Fixed the issue where after the DOUBLE type is converted into the INT type in a query statement, the query result is inaccurate due to an incorrect query range.
• Fixed the issue where when an INSERT operation is performed in a scenario with a large data volume, a minor or major compaction will trigger a retry of the INSERT operation, which may result in discontinuous auto-increment IDs.
• Fixed the issue where during direct load, the memory is used up because data of the ObArray type occupies memory of the sys500 tenant.
• Fixed the issue where when too many sessions exist in the sys tenant, the client may return the error session already exists.
• Fixed the issue where SqlExecutor memory bloats when a tenant with minimum specifications obtains the schemas.
• Optimized the performance in CPU over-allocation scenarios after the cgroup feature is enabled.

Considerations

OceanBase Database Community Edition can be directly upgraded from V4.1.0_CE_BP3 to V4.2.0 GA but not to V4.2.0_CE_Beta.

V4.1.0_CE_BP2

Version information

• Release date: June 15, 2023
• Version: V4.1.0_CE_BP2
• RPM version: oceanbase-ce-4.1.0.1-102000042023061314

Overview

• This version includes compatibility with MySQL's SHOW CREATE TABLE statement to meet the needs of related tools.
• This version addressed several customer and internal testing issues to enhance product stability.

Bug fixes and improvements

• Fixed the issue where querying the TABLE_CONSTRAINTS view would fail when the system variable lower_case_table_names was set to 0.
• Fixed the issue where refreshing the schema after system restart with a large number of partitions caused Proxy to hang when establishing a connection.
• Fixed the issue where data write operations reported error 4017 due to reference count leakage after inserting over 1 billion rows into an unpartitioned table with no primary key.
• Fixed the issue where large transactions involving full table updates continued to report error 4138.
• Fixed compatibility issues with the ObDDLStartLog structure that could cause failures when upgrading from V4.0.0_CE and V4.0.0_CE_BP1 to later versions.
• Fixed the issue where excessive memory was used by SQL Audit during cluster data playback after restart.
• Fixed the issue where abnormal column equal encoding resulted in data exceptions.
• Fixed the issue where core dump might be triggered when timezone info is modified concurrently.
• Added support for complete compatibility with MySQL's SHOW CREATE TABLE statement by setting the session variable _show_ddl_in_compat_mode.

Considerations

• If you are using V4.0.0_CE or V4.0.0_CE_BP1, we recommend that you upgrade directly to V4.1.0_CE_BP2 to avoid possible log playback compatibility issues during the upgrade process.

• If you have unpartitioned tables with large amounts of data and no primary keys, we recommend that you upgrade directly to V4.1.0_CE_BP2 to avoid potential errors that may occur during data write operations.

V4.1.0_CE_BP1_HF1

Version information

• Release date: May 19, 2023
• Version: V4.1.0_CE_BP1_HF1
• RPM version: oceanbase-ce-4.1.0.0-101010022023051821

Overview

This version fixed some bugs and improved stability.

Bug fixes

• Fixed data exceptions caused by inter-column equidistant encoding.
• Fixed the possible data write failures caused by tablet_id change in the TRUNCATE TABLE scenario.

V4.1.0_CE_BP1

Version information

• Release date: May 12, 2023
• Version: V4.1.0_CE_BP1
• RPM version: oceanbase-ce-4.1.0.0-101000022023050809

Overview

This version fixes some bugs and improves stability.

Bug fixes

• Fixed the issue where the error code of the REGEXP BINARY expression was incompatible with that of MySQL.
• Fixed the bug where the SQL execution result sets were duplicated after IN expression optimization.
• Fixed the bug where OceanBase Database deployed in a Kubernetes cluster failed to start due to too many virtual network cards.
• Fixed the bug where the execution plan was not generated as expected after an index was created on a generated column and used for table access.
• Fixed the issue that OceanBase Database does not support the NULLABLE field in the information_schema.STATISTICS view.
• Fixed the bug where plan cache eviction would fail if an SQL execution plan occupies too much memory and contains expressions that cannot be parameterized.
• Fixed the bug related to encoding data estimation error, which resulted in the writing of more than 2 MB of encoded data into the macroblock memory buffer. This bug would result in macroblock serialization failure, which in turn caused the major compaction to be stuck.

Considerations

obd V2.1.0 and later support the upgrade of OceanBase Database from V4.0.0 to V4.1.0.

V4.1.0_CE

Version information

• Release date: April 20, 2023
• Version: V4.1.0_CE
• RPM version: oceanbase-ce-4.1.0.0-100000202023040520

Overview

This version fixes some bugs and improves stability.

Bug fixes

• Fixed the backup failure issue which occurred when many empty tablets exist.
• Fixed the bug where stack overflow would occur when the operating system page size is set to 64 KB.
• Fixed the bug of failing to read the lob data length after the upgrade from version 4.0 to version 4.1.
• Fixed the bug of memory leak caused by the ModulePageAlloc module.
• Fixed the disconnection issue caused by the lack of updating session active time in the response logic of COM_PING.
• Fixed the bug where the OUTPUT_BYTES statistics in the CDB_OB_BACKUP_SET_FILES view was not accurate.

Considerations

• obd V2.0.0 does not support the upgrade of OceanBase Database from V4.0.0 to V4.1.0.
• OCP V4.0.3_CE now supports the upgrade of OceanBase Database from V4.0.0 to V4.1.0.

V4.1.0_CE_BETA

Version information

• Release date: March 20, 2023
• Version: V4.1.0_CE_BETA
• RPM version: oceanbase-ce-4.1.0.0-100000192023032010

Overview

OceanBase Database Community Edition V4.1 leverages the upgraded architecture of V4.0 and is specifically tailored towards public cloud and open-source scenarios. In addition to its rich variety of features, this version is further optimized to achieve improved compatibility with MySQL 8.0, as well as enhanced performance and ease of use at a lower cost. This version is also designed to support large-scale replication, and therefore can be comprehensively commercialized.

This version comes with a range of powerful updates designed to significantly enhance its core capabilities, including:

• Enhancements

  SQL statement-level resource isolation, tenant-level input/output operations per second (IOPS) isolation, and the Latin1 character set are supported.

• Compatibility with MySQL

  Compatibility with MySQL 8.0 in terms of functions, variables, and SQL modes is improved. Geographic information system (GIS) data types are supported.

• Improved performance

  • Transactional processing (TP) performance:

    • Compared with OceanBase Database Community Edition V4.0, our new version improves the comprehensive read/write performance by about 40% when running the Sysbench benchmark with 1024 threads.
    • This version also improves the performance by about 8% in terms of transactions per minute (tpmC) when running the TPC-C benchmark with 1000 warehouses.

  • Analytical processing (AP) performance:

    • Compared with OceanBase Database Community Edition V4.0, our new version improves the TPC-H query performance by about 17% when executing 22 SQL statements sequentially with 100 GB of data.
    • The TPC-DS query performance is improved. This version can execute 99 SQL statements sequentially with 100 GB of data in just 161 seconds.

  • Data import performance: The speed of importing a 74 GB lineitem table through direct load to an OBServer node with the c6a.12xlarge specification (48 vCPUs, 96 GB of memory) can now reach up to 165MB/s.

  • The routing of distributed transactions is optimized based on OceanBase Database Proxy (ODP), improving the performance of distributed transactions.

  • OceanBase Database Community Edition V4.1 also improves the performance of the NESTED LOOP JOIN operator, the execution of Index Skip Scan, the parallel execution of the TRUNCATE TABLE statement, and the compilation feedback.

• Enhanced stability

  The upper limit on the storage size of LOBs is increased.

• Enhanced high availability

  Primary and standby tenants based on archived logs are supported, achieving a recovery time objective (RTO) of less than 8s in more scenarios.

• O&M capabilities

  • Online upgrades of V4.0 series are supported.

  • Statistics are collected in real time.

  • The log system and readability are optimized.

  • Persistence and storage are optimized: Adaptive compactions are supported and space optimization is implemented for small tables.

• Product forms

  A standalone edition is released.

Features

Enhancements

• SQL statement-level resource isolation is supported.

  We noticed that our customers often require more fine-grained control over resource isolation. For example, they use different resource specifications for different SQL statements and isolate resources of different users in the same tenant. This helps application systems allocate and isolate resources for online analytical processing (OLAP) and online transactional processing (OLTP) services, minimizing the mutual impact between services.

  With OceanBase Database Community Edition V4.1, you can create resource groups and bind users to them. When you execute a business SQL statement, the system determines the corresponding resource group based on predefined SQL rules. You can use the DBMS_RESOURCE_MANAGER package to define business resource isolation strategies. Currently, only CPU resource isolation in MySQL mode is supported. For more information, see Configure SQL statement-level resource isolation (Oracle mode) and Configure SQL statement-level resource isolation (MySQL mode).

• Tenant-level IOPS isolation is supported.

  OceanBase Database offers tenant-level isolation since V4.0. That is, resources are distributed among tenants based on their allocated units and I/O categories. This way, each task within a tenant utilizes only its assigned resources and does not compete with other tasks for resources. However, customized resource isolation for some I/O categories based on tasks are not supported. Also, resources cannot be further isolated for different users within the same tenant. To achieve unified resource isolation, we integrated both I/O and CPU resources into the Resource Manager framework on the UI.

  With OceanBase Database Community Edition V4.1, you can customize isolation strategies for I/O resources by using the DBMS_RESOURCE_MANAGER package. Specifically, you can define an isolation plan by setting the MIN_IOPS, MAX_IOPS, and WEIGHT_IOPS parameters in the CREATE_PLAN_DIRECTIVE package function. Note that you can define IOPS isolation plans for both Oracle and MySQL tenants. For more information, see Resource isolation.

• The Latin1 character set is supported.

  As the number of OceanBase Database users continues to grow, we have noticed an increase in demand for the use of the Latin1 character set. By adding support for this character set in OceanBase Database V4.1, we are simplifying project migration that would otherwise require complex character set conversions. In MySQL mode, we support both latin1_bin and latin1_swedish_ci collations, while in Oracle mode, only latin1_bin is supported.

Compatibility with MySQL

• Compatibility with MySQL 8.0 in system functions, variables, and SQL modes is improved.

  Most features of MySQL 5.7 are supported in OceanBase Database V4.0 and earlier versions. As more experienced MySQL users try OceanBase Database public cloud and community editions, we decide to further enhance the compatibility with MySQL 8.0 in OceanBase Database V4.1. Supported features include:

  • System functions
    • Character operation functions: BIN_TO_UUID(), CHARACTER_LENGTH(), LOAD_FILE(), IS_UUID(), UUID_TO_BIN(), and OCTET_LENGTH().
    • Datetime functions: ADDTIME() and DAYNAME().
    • Encryption and decryption functions: DECODE(), DES_DECRYPT(), DES_ENCRYPT(), ENCODE(), and ENCRYPT().
    • Performance information functions: FORMAT_BYTES() and FORMAT_PICO_TIME().
    • Information functions: CURRENT_ROLE() and ICU_VERSION().
    • Analytic functions: BIT_AND(), BIT_OR(), and BIT_XOR().
    • Other functions: NAME_CONST().
  • SQL modes
    • Independent SQL modes: REAL_AS_FLOAT and TIME_TRUNCATE_FRACTIONAL.
    • Composite SQL modes: ANSI, DB2, MAXDB, MSSQL, ORACLE, POSTGRESQL, and TRADITIONAL.
  • Information schemas
    • View compatibility: View fields in OceanBase Database are the same as those in MySQL 8.0, but the field data types, widths, and constraints are adjusted for compatibility. For example, OceanBase Database provides the following views: VIEW_TABLE_USAGE, VIEWS, TABLES, STATISTICS, ENGINES, TABLE_CONSTRAINTS, REFERENTIAL_CONSTRAINTS, CHECK_CONSTRAINTS, PARAMETERS, ROUTINES, and PARTITIONS.
    • Performance improvement: The overall query performance of system views such as STATISTICS is improved based on local table indexes.
  • Float(m, d): OceanBase Database V4.1 is fully compatible with the data types and precisions in MySQL 8.0.

• GIS data types are supported.

  GIS provides storage, computing, and indexing capabilities for spatial objects such as points, lines, planes, and complex spatial objects. GIS is widely applied in the transportation industry and can help quickly build spatial computing capabilities. OceanBase Database supports GIS since V3.2.4 and adopts the quadtree-based indexing solution. We also provide robust support for GIS data types of MySQL 8.0, the management and caching of spatial reference system (SRS) metadata, quadtree-based spatial indexes, single-value types, and multi-value types. Single-value types include GEOMETRY, POINT, LINESTRING, and POLYGON, and multi-value types include MULTIPOINT, MULTILINESTRING, MULTIPOLYGON, and GEOMETRYCOLLECTION. OceanBase Database V4.1 also implements the spatial computing functions commonly used in MySQL 8.0.

  For more information, see Spatial data types.

Improved performance

• The data import performance is improved.

  Data import performance is a crucial metric in the field of OLAP. Applications for data analysis and big data are typically associated with large data volumes. In the AP field, the capability to rapidly import business data from TXT or CSV formats into a database and provide external queries and analysis is a key competitive advantage that influences customers' decision-making.

  When a database system imports data, the SQL layer parses the data received from the client, checks for errors, routes and forwards it to its destination. The data is then put into transactions and stored. However, this process has some limitations: the data is filtered and parsed row by row by the SQL layer, leading to an accumulation of logs that can slow down the system. To address this issue, OLAP databases use the direct load technology to preserve resources and cut costs in SQL processing. By using this method, the data can be saved directly to the disk.

  OceanBase Database V4.1 uses the direct load technology to streamline the data loading path by skipping modules such as the SQL layer, transaction module, and MemTables, to directly persist data to SSTables, thereby significantly accelerating data import. To use the bypass capability, you must add a hint in the LOAD DATA statement, for example, load data /*+ direct(true , 1024) parallel(64) */ infile '/data/1/hits.tsv' into table hits Fields Terminated By '\t';. You can query the V$SESSION_LONGOPS view for the import progress. Primary key conflict and error tolerance processing is supported in different modes. At present, only CSV data files are supported. If you import a 74-GB lineitem table through direct load to an OBServer node with the c6a.12xlarge specification, which provides 48 vCPUs and 96 GB of memory, the import speed can reach 165 MB/s.

• The routing of distributed transactions is optimized based on ODP, improving the performance of distributed transactions.

  During business data modeling, transactions are generally processed on the same data node to avoid compromising the performance due to cross-node access of distributed transactions. In earlier versions of OceanBase Database, only requests in the same transaction can be processed on the same OBServer node. Therefore, ODP cannot route SQL queries based on the locations of data to access. This increases the latency in SQL processing within distributed transactions, thereby compromising the performance. During performance testing, the execution sequence of SQL statements must be adjusted to optimize the performance. Much time and energy are consumed to reduce the negative impact caused by the routing of distributed transactions on the performance.

  OceanBase Database V4.1 optimizes routing for distributed transactions. ODP synchronizes the running status of transactions to OBServer nodes. Not all the running status information needs to be sent to the coordinator. A route can be calculated for each SQL statement. This ensures that SQL requests can be executed on any OBServer node in the cluster, thereby avoiding unnecessary processing latency. You can specify the enable_distributed_route parameter of ODP to enable routing optimization for distributed transactions, which is enabled by default. At present, routing optimization is not supported for eXtended Architecture (XA) transactions.

• Other performance improvements:

  • The performance of the multistage NESTED LOOP JOIN operator is improved by about 10 times in specific Group Rescan scenarios.
  • The execution of Index Skip Scan is optimized. When the NDV operator is used to calculate more than 0.5 million rows of data, the performance can be improved by about 10 times.
  • The parallel execution of the TRUNCATE TABLE statement is optimized, improving the statement execution efficiency by about 10 times.

Enhanced stability

• The upper limit on the storage size of LOBs is increased by using DBMS_LOB.

  In earlier versions of OceanBase Database, the size of a stored LOB is limited to 48 MB. This poses a forcible restriction on the use of LOBs. Through the architectural upgrade in OceanBase Database V4.1, the storage layer splits an LOB macroblock into multiple LOB metadata records for storage. When the LOB macroblock needs to be fetched, these metadata records are aggregated into a continuous buffer, which is then returned to the SQL layer for processing. This breaks the limit on the data storage size and extends the LOB size to 512 MB by using regular SQL statements. In Oracle mode, the LOB size can be extended to the TB level by using the DBMS_LOB package.

Enhanced high availability

• Primary and standby tenants based on archived logs are supported.

  To continuously improve database high availability is one of objectives in the design of OceanBase Database. Based on the distributed architecture and multi-replica data management mechanism, OceanBase Database can meet the high availability requirements of business systems in most scenarios. However, for industries such as banking and insurance that have extremely high requirements on the high availability of databases, when a single cluster is deployed in three IDCs across two regions for geo-disaster recovery, services may still become unavailable due to poor network communication across regions. Therefore, OceanBase Database provides the tenant-level primary/standby solution to resolve the issues in cross-region geo-disaster recovery.

  In earlier versions of OceanBase Database, the primary/standby cluster solution is used based on the cluster-level log transmission strategy. In this strategy, the primary and standby clusters must be able to detect the log synchronization status of each other, and log coupling exists between tenants. In OceanBase Database V4.1, we offer a tenant-level primary/standby solution, where the primary and standby tenants are fully decoupled and unaware of each other. Archive logs are asynchronously synchronized by using a third-party network file system (NFS) or object storage service (OSS). The primary and standby tenants support resumable transmission of logs, switchover, and failover. In the tenant-level primary/standby solution, an OceanBase cluster can contain both primary and standby tenants to help business applications achieve more balanced resource utilization.

  For more information, see Overview of physical standby databases.

• The RTO is reduced to less than 8s in more scenarios.

  Compared with OceanBase Database V4.0, OceanBase Database V4.1 is architecturally upgraded and optimized in fault detection for the distributed architecture and partition election capabilities, aiming to achieve an RTO of less than 8s in most scenarios. Specifically:

  • If the primary OBServer node crashes due to a fault, the standby OBServer node works with ODP to recover services.
  • If the log disk or data disk of the primary OBServer node becomes faulty, the primary node returns an error but does not go offline. The replicas on this node are migrated to a functioning node and a new leader is elected to provide services. In this case, the number of available replicas decreases and the faulty node needs to be manually recovered.

Enhanced O&M capabilities

• Online upgrades of V4.0 series are supported.

  OceanBase Database V4.0 is a major upgrade that introduces significant architectural changes, including updates to storage data, internal tables, views, parameters, and some functionality. Therefore, OceanBase Database cannot be directly upgraded from V3.x to V4.0. We recommend that you use OceanBase Migration Service (OMS) for data logic migration and upgrading instead. However, from V4.1, OceanBase Database provides new capabilities, including system data compatibility, support for physical online upgrades, and uninterrupted project upgrades. These capabilities aim to reduce upgrade complexity and costs while promoting OceanBase Database V4.1 in more scenarios.

  OceanBase Database V4.1 adopts the rotating upgrade strategy.

  1. After an upgrade is initiated, the cluster enters the hybrid deployment state. The system replaces the binary programs of OBServer nodes by zone and exits the hybrid deployment state until the binary programs of all nodes are replaced. At this time, binary programs of the later database version host the data and behaviors of the earlier version in compatibility mode.
  2. After you run an upgrade command in a tenant, the system verifies and corrects the variables, system tables, and data. You must run the same upgrade command for all the tenants in the cluster one by one to finish the overall upgrade of the cluster. We recommend that you use OceanBase Cloud Platform (OCP) to upgrade your cluster online.

  OceanBase Database V4.1 supports the following upgrade scenarios:

  • Online cluster upgrades
  • Primary/Standby cluster upgrades. You can upgrade the primary and standby clusters separately. We recommend that you upgrade the standby cluster before the primary cluster. Before the upgrade, use the switchover command to adjust the state of the tenants in the cluster to ensure that they are in the consistent state.
  • Continuous upgrades since OceanBase Database V4.0, including the upgrades of features and patches.
  • Tenant-level upgrades. We recommend that you upgrade all tenants in the cluster to the same version.
  • Note that upgraded OBServer nodes do not support rollbacks. Therefore, we recommend that you back up the data and binary programs before the upgrade.

  Do not perform the following operations during the upgrade, which are allowed after the upgrade is completed:

  • DDL operations
  • Major freezes
  • Migration, replication, and load balancing
  • Physical backup and restore
  • Switchover and failover
  • Tenant creation

• The log system and readability are optimized.

  OceanBase Database V4.1 systematically analyzes and optimizes log generation in the following aspects:

  • Definition of log levels: The DIAG log level is added. WARN and ERROR logs that are imperceptible to users are classified as DIAG logs. The levels are adjusted for O&M-oriented WARN and ERROR logs that need to be retained.

    Current log level	Intended audience	Definition	Original log level
    ERROR	DBA or general users	Exceptions that cause the OBServer node to be unable to provide normal services. For example, the disk is full or the listening port is occupied. For another example, an internal check error is returned in the background, such as 4377(dml defensive check error) or 4103 (data checksum error). In this case, manual intervention is needed.	/
    WARN	DBA or general users	Unexpected scenarios where the OBServer node can provide services but its behaviors are not as expected. For example, a write throttling alert is generated.	/
    INFO	DBA or general users	Information about system changes.	INFO
    EDIAG	R&D engineers	Error diagnosis information indicating unexpected logic errors, such as unexpected function arguments.	ERROR
    WDIAG	R&D engineers	Warning diagnosis information indicating expected errors, such as failures returned by functions.	WARN
    TRACE	R&D engineers	Task-level debugging information.	TRACE
    DEBUG	R&D engineers	Detailed debugging information.	DEBUG

  • Error code association: Error messages are classified and error codes are improved. A response error code is defined for each warning and error. You can retrieve documents based on an error code to find the solution.

  • Throttling by error code: WDIAG logs are throttled by error code. For throttled logs, only the error code and the number of throttled logs are displayed. The diag_syslog_per_error_limit parameter is added for controlling this throttling feature. By default, 200 DIAG logs per second are allowed for each error.

  • Log file self-description: The server address, host type, CPU type, operating system version, OBServer version, and time zone are recorded in the header of each log file.

• Views are standardized. Views and internal tables are redesigned. Totally 73 standard views are provided. We recommend that you query views for information such as the metadata of OceanBase Database.

• Statistics are collected in real time.

  The accuracy of statistics is the basis for the optimizer to generate a proper execution plan. Earlier versions of OceanBase Database already support statistics collection in the following ways:

  • Use the DBMS_STATS package or the ANALYZE command to manually trigger statistics collection.
  • Configure scheduled tasks to automatically collect statistics.
  • Automatically collect statistics when the data amount exceeds the specified threshold.

  OceanBase Database V4.1 supports online statistics collection. You can enable this feature by using the _optimizer_gather_stats_on_load system variable or hints. If the business system supports the CTAS or INSERT statement, OceanBase Database updates incremental statistics in real time. This avoids system overheads caused by full table scans performed in regular statistics collection.

• Data persistence and storage are optimized.

  • Adaptive compaction: A compaction consumes a large number of I/O operations and CPU resources. Therefore, different compaction strategies are needed in different business scenarios. In the log-structured merge (LSM) storage system, the core objective of compactions is to achieve an optimal balance among write amplification, read amplification, and space amplification. In the adaptive compaction strategy, the system collects statistics about and analyzes the historical minor compactions and query operations performed on a table to extract the characteristics of the table, determines whether to schedule a major compaction for this table based on these characteristics, and completes a major compaction that is imperceptible to users.

  • Table space optimization: If a small table is split into many partitions, storage amplification appears in macroblock storage. This is prone to result in insufficient disk space when a large number of partitions exist. Multiple small SSTables are stored in a physical macroblock for compaction and storage. This achieves a utilization of about 4% and a fragment ratio of about 1.5% for the macroblock when a small table is split into many partitions, significantly improving the disk utilization.

Product forms

• A standalone edition is released.

  Based on the integrated architecture for standalone and distributed modes of OceanBase Database V4.0, we released a standalone edition for OceanBase Database V4.1. The standalone edition can be deployed by using OCP or OceanBase Deployer (obd), enabling independent software vendors (ISVs), small-sized customers, and community users to quickly get started with the new features of OceanBase Database V4.1. If the standalone edition can no longer meet your project requirements, business requirements, or high availability requirements, you can upgrade the standalone edition to a distributed edition online with ease.

  For more information, see Deploy OceanBase Database on a single OBServer node.

Performance test report

Test environment specifications

Version for test

Sysbench OLTP benchmark

Test plan

1. Use obd to deploy an OceanBase cluster. Install ODP on a separate server with a specification of ecs.c7.16xlarge, which provides 64 CPU cores and 128 GB of memory, and install Sysbench tools on another server to avoid resource contention.
2. Deploy the cluster in the 1-1-1 architecture, where the cluster has three zones and each zone has one OBServer node. After successful deployment, create the tenant and users required for running the Sysbench benchmark. The sys tenant is a built-in system tenant used for managing the cluster. Do not use the sys tenant to run the benchmark. Set primary_zone to RANDOM for the tenant, which indicates that the leader of new table partitions is randomly assigned to one of the three servers.
3. Launch the Sysbench client and perform the point_select, read_write, read_only, and write_only tests.
4. Set the --time parameter to 60s for each round of test. The number of threads can be 32, 64, 128, 256, 512, or 1,024.
5. The test dataset consists of 30 non-partitioned tables with 1 million rows of data. To be specific, set --tables to 30 and --table_size to 1000000.

Tenant specifications

• MAX_CPU = 26
• MEMORY_SIZE = 70 GB

Test results

Point select performance

Read-only performance

Write-only performance

Read/Write performance

TPC-C benchmark test with BenchmarkSQL

Test plan

1. Use obd to deploy an OceanBase cluster. Deploy ODP and the TPC-C tools on the same server to avoid insufficient stress on the client.
2. Deploy the OceanBase cluster in the 1-1-1 architecture, where the cluster has three zones and each zone has one OBServer node. After successful deployment, create the tenant and users required for running the TPC-C benchmark. The sys tenant is a built-in system tenant for managing the cluster. Do not use the sys tenant to run the benchmark. Set primary_zone to RANDOM for the tenant.

Tenant specifications

• MAX_CPU = 26
• MEMORY_SIZE = 70 GB

Software versions

mysql-connector-java-5.1.47

BenchmarkSQL V5.0

Test configuration

• warehouses = 1000
• loadWorkers = 40
• terminals = 800
• runMins = 5
• newOrderWeight = 45
• paymentWeight = 43
• orderStatusWeight = 4
• deliveryWeight = 4
• stockLevelWeight = 4

Test results

TPC-H benchmark test

Test plan

1. Use obd to deploy an OceanBase cluster. Deploy the TPC-H client on a server for stress testing. You do not need to deploy ODP. You can connect to any server during testing.
2. Deploy the OceanBase cluster in the 1-1-1 architecture, where the cluster has three zones and each zone has one OBServer node. After successful deployment, create the tenant and users required for running the TPC-H benchmark. The sys tenant is a built-in system tenant for managing the cluster. Do not use the sys tenant to run the benchmark. Set primary_zone to RANDOM for the tenant. Load the data, sequentially execute 22 SQL queries, and take the average value.
3. Size of the test dataset: 100 GB.

Tenant specifications

• MAX_CPU = 26
• MEMORY_SIZE = 70 GB

Test results

TPC-DS benchmark test

Test plan

1. Use obd to deploy an OceanBase cluster. Deploy the client on a separate server for stress testing. You do not need to deploy ODP. You can connect to any server during testing.
2. Deploy the OceanBase cluster in the 1-1-1 architecture, where the cluster has three zones and each zone has one OBServer node. After successful deployment, create the tenant and users required for running the TPC-DS benchmark. The sys tenant is a built-in system tenant for managing the cluster. Do not use the sys tenant to run the benchmark. Set primary_zone to RANDOM for the tenant. Load the data, sequentially execute 99 SQL queries, and take the average value.
3. Size of the test dataset: 100 GB.

Tenant information

• MAX_CPU = 26
• MEMORY_SIZE = 70 GB

Test results

Compatibility changes

Product behavioral changes

Views, entity tables, and virtual tables

For more information, see System views.

Parameter changes

Considerations

• An online upgrade from OceanBase Database V3.x to OceanBase Database V4.x is not supported.
• You can use OMS to migrate the data logic of OceanBase Database V3.x to OceanBase Database V4.1.
• An online upgrade from OceanBase Database V4.0 to OceanBase Database V4.1 is supported.

Recommended versions of tools

The following table describes the recommended component versions for OceanBase Database V4.1.0.

Limitation

• In OceanBase Database V4.1.0, load balancing is not supported after replica scale-out. Therefore, data that already exists before the scale-out cannot be distributed to the newly added nodes. This issue will be resolved in V4.2.
• OceanBase Database V4.1.0 does not support replicated tables. This issue will be resolved in V4.2.
• OceanBase Database V4.1.0 allows you to initiate data backup from the primary cluster. You will be able to initiate data backup from the standby cluster in later versions.
• In OceanBase Database V4.1.0, the minimum specification of a node in the production environment is 4C16GB. Each tenant requires more than 2 GB of memory and more than 1 CPU core.
• OceanBase Database V4.1.0 supports only full-featured replicas. Other types of replicas such as read-only replicas will be supported in later versions.
• In OceanBase Database V4.1.0, you can only increase the number of resource units (UNIT_NUM) of a tenant. You will be allowed to decrease the number of resource units in later versions.
• In OceanBase Database V4.1.0, if you execute the DROP DATABASE statement to drop more than 200 tables from a database at a time, the operation may fail. We recommend that you first execute DROP TABLE and then DROP DATABASE. This issue will be resolved in V4.2.