OBLOADER & OBDUMPER V5.0.0 adds support for skipping records at the beginning and end of files. It improves parallel export, credential management for multiple accounts, and object definition export in OceanBase Database Oracle mode. This release also fixes issues with file processing, data compatibility, and runtime diagnostics, and upgrades dependencies.
Performance improvements
OBLOADER & OBDUMPER V5.0.0 improves import and export performance across different scenarios. The following table summarizes the improvements.
The results below use V4.3.7 as the baseline.
Category |
Metric |
Improvement |
Test conditions |
|---|---|---|---|
| INSERT | Standard import throughput | Up to 3.75 times the baseline | ClickBench, 1 million rows, 4 GB heap |
| INSERT | Import throughput with a small heap | Up to 38.73 times the baseline | ClickBench, 4 million rows, 1 GB heap; resolves the bottleneck caused by sustained memory throttling |
| INSERT | Cumulative CPU consumption on the OceanBase Database side | Reduced by 27.16% | ClickBench, 1 million rows, 4 GB heap, for the same import workload |
| Load | Direct Load throughput | 1.47 times the baseline | ClickBench, 20 GB, 4 GB heap |
| Load | Cumulative CPU consumption on the OceanBase Database side | Reduced by 7.84% | ClickBench, 20 GB, 4 GB heap, for the same import workload |
| Dump | CSV export throughput | Up to 2.18 times the baseline | Approximately 1 GB of data, 4 GB heap |

INSERT: Standard import throughput reaches up to 3.75 times the baseline
Improved read/write scheduling, default batching strategies, and reuse of binding plans within batches. These changes reduce read waits, repeated construction work, and write overhead, improving standard import performance with the default configuration.
Test conditions |
V4.3.7 elapsed time |
V5.0.0 elapsed time |
Speedup |
|---|---|---|---|
| 1 million rows, 4 GB heap | 73.13 seconds | 19.48 seconds | 3.75 times |
Load: Throughput for a 20 GB Direct Load reaches 1.47 times the baseline
Improved batching and budgeting for in-flight data in Direct Load, reducing data backlog and client memory usage when importing large files.
Test conditions |
V4.3.7 elapsed time |
V5.0.0 elapsed time |
Speedup |
|---|---|---|---|
| 20 GB, 4 GB heap | 493.73 seconds | 336.89 seconds | 1.47 times |
Resource metric |
V4.3.7 |
V5.0.0 |
Change |
|---|---|---|---|
| Peak client RSS | 20.21 GiB | 3.47 GiB | Approximately one-sixth of the baseline (17.17%) |
RSS includes resident memory-mapped pages and is not equivalent to Java heap memory. A reduction in cumulative CPU consumption does not imply a lower instantaneous CPU load during import.
Dump: CSV export throughput reaches up to 2.18 times the baseline
Reusing row buffers and reducing initialization and object allocation overhead improves export efficiency for CSV and other formats.
Source CSV size |
V4.3.7 elapsed time |
V5.0.0 elapsed time |
Speedup |
|---|---|---|---|
| Approximately 1 GB (measured: 0.99 GB) | 64.07 seconds | 29.41 seconds | 2.18 times |
Resolving memory throttling bottlenecks: Throughput reaches up to 38.73 times the baseline
In earlier versions, limited memory and a backlog of in-flight data in small-heap environments could repeatedly trigger throttling for memory protection. V5.0.0 redesigns read/write scheduling and task-level memory budgets to address this issue. It limits in-flight data based on available memory, reduces the rate of reading as memory pressure rises, and prioritizes draining existing batches. This prevents a persistent backlog from compounding repeated throttling. In tests with a 1 GB heap, these changes resolved the throughput bottleneck caused by sustained memory throttling in the earlier version, allowing concurrent writes to be effective even with a small heap.
Performance shows little regression with a 1 GB heap compared with a 4 GB heap. In the same-version test with 4 million rows, the 1 GB configuration used one-quarter of the heap memory while retaining 97.46% of the throughput of the 4 GB configuration, substantially reducing the dependence on heap size.
The following chart compares elapsed times for a 1-million-row INSERT test. With a 1 GB heap, V5.0.0 achieved 15.83 times the throughput of V4.3.7 in this test. The 38.73-fold result above applies to the 4-million-row test.

Cumulative CPU consumption on the OceanBase Database side
In addition to improved client processing performance, OceanBase Database consumes less cumulative CPU time to complete the same import workload: 27.16% less for INSERT with 1 million rows, and 7.84% less for Direct Load with 20 GB of data.
Test conditions |
V4.3.7 cumulative CPU consumption |
V5.0.0 cumulative CPU consumption |
Reduction |
|---|---|---|---|
| INSERT, 1 million rows, 4 GB heap | 228.79 CPU-seconds | 166.65 CPU-seconds | 27.16% |
| Direct Load, 20 GB, 4 GB heap | 2471.13 CPU-seconds | 2277.48 CPU-seconds | 7.84% |

Adaptive Reader/Writer scaling and dynamic read/write ratios
V5.0.0 enables adaptive Reader/Writer scheduling and task-level budgets for in-flight data by default. The numbers of Readers and Writers can increase or decrease automatically as runtime conditions change. The read/write ratio adjusts accordingly to match the file read rate to the database's write capacity.
Dynamic read and write concurrency: Reader and Writer counts are adjusted independently based on the read rate, write throughput, database response, CPU usage, and heap memory pressure. The read/write ratio is then rebalanced to reduce waits and backlogs caused by insufficient reads or slow consumption.
Prioritizing draining under memory pressure: Task-level budgets limit the memory occupied by in-flight batches. As memory pressure rises, the scheduler reduces the read rate, adjusts concurrency, and prioritizes writing batches that have already been generated. This reduces backlogs and repeated throttling for memory protection.
Less manual tuning: Standard INSERT workloads no longer require repeated experimentation with thread counts, fixed read/write ratios, buffer capacity, or batch sizes. Tasks adapt to the current workload across different resource configurations.
File parsing improvements
Improved parsing and read paths for multiple file formats, reducing parsing and object allocation overhead and enhancing parallel reads of a single file.
Single-threaded parsing throughput increased by 71.33%. In the dedicated single-threaded CUT test, throughput increased from 64.56 MiB/s to 110.61 MiB/s. These are measured single-threaded results using the same test baseline as the table below.
Format |
Concurrency |
Parsing throughput improvement |
|---|---|---|
| CSV | 16 threads → 16 threads | 155.77% |
| CUT | Single thread → single thread | 71.33% |
| CUT | 32 threads → 32 threads | 56.76% |
| SQL | 16 threads → 16 threads | 74.02% |
| POS | 16 concurrent readers → 16 concurrent readers | 56.88% |
| Avro | Actual read concurrency: 1 → 16 | Up to 15.43 times the baseline |
| Parquet | Actual read concurrency: 1 → 16 | Up to 13.40 times the baseline |
New features
Added the
--skip-header Nand--skip-footer Noptions to skip the specified number of records at the beginning and end of each source file, respectively, when importing CSV, CUT, or POS files.Improved parallel export. SAMPLE is now the default split strategy. Use
--split-type LEGACYto select the previous strategy.Enhanced credential management in
secure-gento support multiple accounts. A single encrypted credential file can now store credentials for multiple application and sys accounts. Credentials are matched by user, tenant, cluster, and address, simplifying connections across environments without manual password entry.Improved object definition export in OceanBase Database Oracle mode. Added support for exporting comments on views and view columns, and improved the preservation of metadata for partitioned unique indexes.
Bug fixes
File and script processing
Fixed issues with explicit column mappings for CSV files, MySQL column name matching, and control file lookup that could cause values to be imported into the wrong columns.
Fixed a split processing performance regression when filtering records at the beginning and end of files.
Fixed an issue where valid U+FEFF characters at the beginning of CUT file splits were lost.
Fixed issues with short reads, final lines without line breaks, record lengths, and multibyte boundary handling in POS files. Data with invalid encoding is now handled as bad records.
Fixed issues with changes to column lists and mixed use of explicit and implicit column lists in SQL files. Improved batch error handling to avoid affecting subsequent valid records.
Improved statement boundary detection and JDBC escape rewriting for DDL/MIX files containing comments, DELIMITER directives, Oracle blocks, or q-quote syntax. Session state and transaction order are preserved within MIX files.
Data type and object compatibility
Fixed NLS conversion errors when importing native ORC timestamps into OceanBase Database in Oracle mode, and issues with preserving DECIMAL precision.
Improved handling of time, Decimal, NULL, and nested types in Parquet/Avro files. Fixed an issue where batches containing valid Avro union data were rejected in their entirety.
Fixed issues with exporting MySQL TIME values, including loss of microsecond precision and failure to preserve values exceeding 24 hours.
Fixed an issue where records with parameter binding failures were still included in the successful row count.
Fixed index and constraint name conflicts and incorrect rewriting of special identifiers during imports with remapping.
Fixed foreign key dependency ordering when importing parent and child tables through standard JDBC into OceanBase Database 4.x in Oracle mode.
Fixed an issue where hidden primary keys were exported for tables that already had explicitly defined primary keys.
Disabled LOB Locator V2 by default to improve LOB import and export compatibility.
Error handling and runtime diagnostics
Fixed issues where export tasks hung or were incorrectly marked as complete when the disk was full, file rollover failed, or finalization failed. Improved error propagation when merging parallel export output into a single file and initializing CSV headers.
Fixed incorrect parsing of local paths containing special characters such as
#,?, and%, as well as issues with output directory validation and inaccurate error messages.Restored periodic progress logging and corrected the reporting of throughput, buffer capacity, and wait time.
Fixed an issue where a connection was returned even after its initialization SQL failed to execute.
Fixed an issue where trustStore configuration was still required for SSL connections configured with
trustServerCertificate=true.
Security fixes
Upgraded Guava and addressed the related deserialization risks in Hadoop's relocated namespaces.
Updated dependencies, including Log4j, Netty, and the Jackson library embedded in Parquet, to address related security risks.
Removed the legacy SOFA Common and Commons Lang 2 runtime dependencies.
Dependency upgrades
Upgraded Guava from 32.0.1-jre to 33.7.2-jre.
Upgraded Log4j from 2.18.0 to 2.25.5.
Upgraded Netty from 4.1.136.Final to 4.1.137.Final.
Upgraded AWS SDK v2 from 2.35.4 to 2.54.14.
Upgraded obkv-table-client from 2.4.0 to 2.4.1.
Upgraded parquet-jackson from 1.15.1 to 1.18.1.