Background information
Incremental direct load uses REPLACE semantics, which means that it does not check for primary key conflicts and directly replaces rows with duplicate primary keys. For more information, see Data import overview. To synchronize the imported data to downstream services, obcdc must adapt to the semantics of incremental direct load.
Notice
obcdc can synchronize data imported by using incremental direct load but not data imported by using full direct load.
Limitations
obcdc cannot synchronize large object (LOB) data stored in OUTROW mode.
Prerequisites
You have deployed obcdc V4.3.1 or later.
You have deployed OceanBase AI Database.
Procedure
Assume that tables A and B exist in the OceanBase AI Database cluster. The A table contains the following data, with the col1 column as the primary key:
+------+------+
| col1 | col2 |
+------+------+
| 1 | 11 |
| 2 | 22 |
+------+------+
The B table contains the following data, with the col1 column as the primary key:
+------+------+
| col1 | col2 |
+------+------+
| 1 | 21 |
| 3 | 33 |
+------+------+
The following example shows how to use obcdc to import data from table B into table A in incremental direct load mode.
Modify the configuration file.
In the configuration file, set
enable_direct_load_incto1to enable incremental direct load synchronization. For more information about the parameters in the configuration file, see obcdc parameters.enable_direct_load_inc=1Apply the configuration.
A restart is required for the
enable_direct_load_incparameter to take effect. You can apply this configuration in one of the following ways depending on how you use obcdc.If you use obcdc in your own data consumption pipeline, compile and run your code again. For more information, see obcdc development instructions.
If you use obcdc through obcdc_tailf, run the obcdc_tailf command again. Example:
[admin@test oceanbase]$ ./bin/obcdc_tailf -f libobcdc.conf -D output.txtIn this example,
libobcdc.confis the relative path of the configuration file, andoutput.txtis the file that stores the output data. Replace the configuration file and output data file as needed. For more information about theobcdc_tailfcommand, see obcdc_tailf.
Log on to OceanBase AI Database and import data in incremental direct load mode.
obclient [test]> insert /*+ enable_parallel_dml parallel(2) direct(true, 0, 'inc_replace') */ into A select * from B;For more information about the command, see Import data by using the INSERT INTO SELECT statement.
View the execution result.
obclient [test]> SELECT * FROM A;The following output indicates that data is imported from table
Binto tableA:+------+------+ | col1 | col2 | +------+------+ | 1 | 21 | | 2 | 22 | | 3 | 33 | +------+------+View the output of obcdc.
After data is imported in incremental direct load mode in OceanBase AI Database, obcdc outputs the corresponding information to downstream components. A simplified example is as follows:
[EBGIN]; [EPUT] column_name:col1, old_value:1, new_value:1, column_name: col2, new_value: 21; [EPUT] column_name:col1, old_value:3, new_value:3, column_name: col2, new_value: 33; [ECOMMIT];