This topic describes how to use obloader & obdumper to migrate data from an Oracle tenant to a MySQL tenant in OceanBase Database.
About obloader & obdumper
OceanBase Database provides the data export tool obdumper and the data import tool obloader.
obdumper is a client data export tool developed in Java. You can use it to export data and the DDL statements of objects in OceanBase Database to files. For more information about obdumper, see obdumper overview.
obloader is a client data import tool developed in Java. obloader provides extensive command-line options that allow you to import definitions and data to OceanBase Database in many complex scenarios. For more information about obloader, see obloader overview.
Procedure
Prepare the running environment for obdumper and obloader. For more information, see Prepare the environment and Download tools.
Create a directory to store the exported data.
Export the table schema and data.
[xxx@xxx /ob-loader-dumper-4.0.0-RELEASE/bin] $./obdumper -h <host IP address> -P <port number> -u <username> -p <password> [--sys-user <username in the sys tenant> --sys-password <password of the specified user in the sys tenant>] -c <cluster name> -t <tenant> -D <schema name> [--ddl] [--csv|--sql] [--all|--table 'table name'] -f <data file or directory>Note
- When you use the host IP address and port number of a physical node to export data, you do not need to specify the
-coption. - For more information about obdumper command-line options, see Command-line options.
- When you use the host IP address and port number of a physical node to export data, you do not need to specify the
Import a table schema and data.
[xxx@xxx /ob-loader-dumper-4.0.0-RELEASE/bin] $./obloader -h <host IP address> -P <port number> -u <username> -p <password> --sys-user <root or proxyro user in the sys tenant> --sys-password <password of the user in the sys tenant> -c <cluster> -t <tenant> -D <schema name> [--ddl] [--csv|--sql] [--all|--table 'table name'] -f <data file or directory>Note
- When you use the host IP address and port number of a physical node to import data, you do not need to specify the
-coption. - For more information about obloader command-line options, see Command-line options.
- When you use the host IP address and port number of a physical node to import data, you do not need to specify the
Examples
The following table describes the database information that is used in the examples.
Database information |
Example value |
|---|---|
| Cluster name | test4000 |
| IP address of the OceanBase Database Proxy (ODP) host | xxx.xxx.xxx.xxx |
| ODP port number | 2883 |
| Password of the root user in the sys tenant | ****** |
| Source tenant name (Oracle mode) | oracle001 |
User account of the oracle001 tenant (with read and write privileges) |
obdumper_user01 |
Password of the obdumper_user01 user in the oracle001 tenant |
****** |
Schema name in the oracle001 tenant |
OBDUMPER_USER01 |
| Destination tenant name (MySQL mode) | mysql001 |
User account of the mysql001 tenant (with the read and write privileges) |
obloader_user01 |
Password of the obloader_user01 user in the mysql001 tenant |
****** |
Schema name in the mysql001 tenant |
test_data |
Migrate a table schema
When you use obdumper and obloader to migrate a table schema, you must specify the --sys-user and --sys-password options to obtain the metadata of the table schema. If you do not specify these options, data export/import features and performance may be significantly affected.
Note
The --sys-user option is used to connect to a user with specific privileges in the sys tenant. If you do not specify the --sys-user option, --sys-user root is used by default.
Export DDL definition files
Scenario: Export all supported object definition statements from the OBDUMPER_USER01 schema of the oracle001 tenant in the test4000 cluster to the /home/admin/test_migrate_data/ddl_data directory.
The sample statement is as follows:
[xxx@xxx /ob-loader-dumper-4.0.0-RELEASE/bin] $./obdumper -h xxx.xxx.xxx.xxx -P 2883 -u obdumper_user01 -p ****** -c test4000 -t oracle001 -D obdumper_user01 --sys-user root --sys-password ****** --ddl --all -f /home/admin/test_migrate_data/ddl_data 2022-12-26 18:41:07 [INFO] Parsed args: [--ddl] true [--file-path] /home/admin/test_migrate_data/ddl_data [--host] xxx.xxx.xxx.xxx [--port] 2883 [--user] obdumper_user01 [--tenant] oracle001 [--cluster] test4000 [--password] ****** [--database] obdumper_user01 [--sys-user] root [--sys-password] ****** [--all] true 2022-12-26 18:41:07 [INFO] Load jdbc driver class: "com.oceanbase.jdbc.Driver" finished 2022-12-26 18:41:07 [INFO] The manifest file: "/home/admin/test_migrate_data/ddl_data/data/MANIFEST.bin" has been saved 2022-12-26 18:41:07 [INFO] Query column metadata for the table: "TEST_TBL1" finished 2022-12-26 18:41:07 [INFO] Query column metadata for the table: "TEST_TBL2" finished 2022-12-26 18:41:07 [INFO] Found 2 empty tables before dump out records. Elapsed: 11.61 ms 2022-12-26 18:41:07 [WARN] No views are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No triggers are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No functions are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No procedures are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No types are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No type bodies are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No packages are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No package bodies are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No synonyms are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No public synonyms are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No sequences are exist in the schema: "OBDUMPER_USER01" 2022-12-26 18:41:07 [WARN] No table groups are exist in the tenant: "oracle001" 2022-12-26 18:41:07 [INFO] Generate 1 dump tasks finished. Total Elapsed: 3.876 ms 2022-12-26 18:41:07 [INFO] Start 3 schema dump threads for 1 dump tasks finished. 2022-12-26 18:41:07 [INFO] Build direct com.alibaba.druid.pool.DruidDataSource finished 2022-12-26 18:41:07 [INFO] No need to acquire DataSource for xxx@sys, as observer is 4.0.0.0 2022-12-26 18:41:07 [INFO] Return the latest compatible version: 4.0.0.0 -> 4.0.0.0 2022-12-26 18:41:07 [INFO] Dump create objects success. DbType: OBORACLE Version: 4.0.0.0 2022-12-26 18:41:07 [INFO] ObOracle(4.0.0.0) is older than 4.0.0.0 ? false 2022-12-26 18:41:07 [INFO] Load meta/oboracle/oboracle22x.xml, meta/oboracle/oboracle40x.xml successed 2022-12-26 18:41:07 [INFO] Query 0 dependencies elapsed 41.82 ms 2022-12-26 18:41:08 [INFO] Query table: "TEST_TBL1" attr finished. Remain: 0 2022-12-26 18:41:08 [INFO] Query table: "TEST_TBL2" attr finished. Remain: 0 2022-12-26 18:41:08 [INFO] Query 2 tables elapsed 285.4 ms 2022-12-26 18:41:08 [INFO] Dump [TABLE] TEST_TBL1 to "/home/admin/test_migrate_data/ddl_data/data/OBDUMPER_USER01/TABLE/TEST_TBL1-schema.sql " finished 2022-12-26 18:41:08 [INFO] Dump [TABLE] TEST_TBL2 to "/home/admin/test_migrate_data/ddl_data/data/OBDUMPER_USER01/TABLE/TEST_TBL2-schema.sql " finished 2022-12-26 18:41:08 [INFO] No.1 It has dumped 2 TABLEs finished. Remain: 0 2022-12-26 18:41:08 [INFO] Total dumped 2 TABLEs finished. Elapsed: 376.1 ms 2022-12-26 18:41:08 [INFO] Dump the ddl of schema: "OBDUMPER_USER01" finished 2022-12-26 18:41:08 [INFO] Close connection count: 19 of the DataSource. Key: xxx.xxx.xxx.xxx_11532_840547833_obdumper_user01 2022-12-26 18:41:08 [INFO] Shutdown task context finished 2022-12-26 18:41:08 [INFO] ---------- Finished Tasks: 1 Running Tasks: 0 Progress: 100.00% ---------- 2022-12-26 18:41:08 [INFO] All Dump Tasks Finished: ---------------------------------------------------------------------------------------------------------------------------- No.# | Type | Name | Count | Status ---------------------------------------------------------------------------------------------------------------------------- 1 | TABLE | TEST_TBL2 | 1 | SUCCESS 2 | TABLE | TEST_TBL1 | 1 | SUCCESS ---------------------------------------------------------------------------------------------------------------------------- Total Count: 2 End Time: 2022-12-26 18:41:08 2022-12-26 18:41:08 [INFO] Dump schema finished. Total Elapsed: 1.070 s 2022-12-26 18:41:08 [INFO] Unnecessary to upload the data files to the remote cloud storage service 2022-12-26 18:41:08 [INFO] System exit 0The directory structure exported is as follows:
[xxx@xxx /ob-loader-dumper-4.0.0-RELEASE/bin] $tree /home/admin/test_migrate_data/ddl_data /home/admin/test_migrate_data/ddl_data ├── data │ ├── CHECKPOINT.bin │ ├── MANIFEST.bin │ └── OBDUMPER_USER01 │ └── TABLE │ ├── TEST_TBL1-schema.sql │ └── TEST_TBL2-schema.sql └── logs ├── ob-loader-dumper.error ├── ob-loader-dumper.info └── ob-loader-dumper.warn 4 directories, 7 filesView the exported files.
[xxx@xxx /ob-loader-dumper-4.0.0-RELEASE/bin] $cat /home/admin/test_migrate_data/ddl_data/data/OBDUMPER_USER01/TABLE/TEST_TBL1-schema.sql CREATE TABLE "TEST_TBL1" ( "COL1" NUMBER(11,0) NOT NULL, "COL2" VARCHAR2(20 BYTE), "COL3" NUMBER(11,0), CONSTRAINT "TEST_TBL1_OBPK_1672051192536124" PRIMARY KEY ("COL1") );
Import DDL definition files
Scenario: Export all supported definitions from the OBDUMPER_USER01 schema of the oracle001 tenant under the test4000 cluster to the /home/admin/test_migrate_data/ddl_data directory, and import it into the test_data schema of the mysql001 tenant under the test4000 cluster.
Notice
DDL files exported from an Oracle tenant do not meet the syntax requirements of a MySQL tenant. Therefore, before you import the DDL files to the MySQL tenant, you must first modify the files so that the files meet the syntax requirements of the MySQL tenant.
For example, for a DDL file exported from the oracle001 tenant, make the following modifications:
- Convert the data types to those supported in MySQL mode:
- Convert
NUMBER(11,0)toint(11). - Convert
VARCHAR2(20 BYTE)tovarchar(20).
- Convert
- Remove the
CONSTRAINT "TEST_TBL1_OBPK_1672051192536124"parameter. - Remove all double quotation marks.
The sample statement is as follows:
[xxx@xxx /ob-loader-dumper-4.0.0-RELEASE/bin] $./obloader -h xxx.xxx.xxx.xxx -P 2883 -u obloader_user01 -p ****** --sys-user root --sys-password ****** -c test4000 -t mysql001 -D test_data --ddl --all -f /home/admin/test_migrate_data/ddl_data 2022-12-27 11:00:35 [INFO] Parsed args: [--ddl] true [--file-path] /home/admin/test_migrate_data/ddl_data [--host] xxx.xxx.xxx.xxx [--port] 2883 [--user] obloader_user01 [--tenant] mysql001 [--cluster] test4000 [--password] ****** [--database] test_data [--sys-user] root [--sys-password] ****** [--all] true 2022-12-27 11:00:35 [INFO] Load jdbc driver class: "com.oceanbase.jdbc.Driver" finished 2022-12-27 11:00:35 [INFO] The manifest file: "/home/admin/test_migrate_data/ddl_data/data/MANIFEST.bin" has been saved 2022-12-27 11:00:35 [INFO] Init writer thread pool finished 2022-12-27 11:00:35 [WARN] The object type : "SEQUENCE" doesn't exist in the -schema.sql files 2022-12-27 11:00:35 [WARN] The object type : "TABLE_GROUP" doesn't exist in the -schema.sql files 2022-12-27 11:00:35 [INFO] Start 128 schema file loader threads successed 2022-12-27 11:00:36 [INFO] No.1 sql of the file: "/home/admin/test_migrate_data/ddl_data/data/OBDUMPER_USER01/TABLE/TEST_TBL1-schema.sql" exec success. Elapsed: 70.51 ms 2022-12-27 11:00:36 [INFO] Load file: "TEST_TBL1-schema.sql" succeeded 2022-12-27 11:00:36 [INFO] No.1 sql of the file: "/home/admin/test_migrate_data/ddl_data/data/OBDUMPER_USER01/TABLE/TEST_TBL2-schema.sql" exec success. Elapsed: 118.7 ms 2022-12-27 11:00:36 [INFO] Load file: "TEST_TBL2-schema.sql" succeeded 2022-12-27 11:00:36 [WARN] The object type : "VIEW" doesn't exist in the -schema.sql files 2022-12-27 11:00:36 [WARN] The object type : "FUNCTION" doesn't exist in the -schema.sql files 2022-12-27 11:00:36 [WARN] The object type : "PROCEDURE" doesn't exist in the -schema.sql files 2022-12-27 11:00:36 [WARN] The object type : "TRIGGER" doesn't exist in the -schema.sql files 2022-12-27 11:00:36 [WARN] The object type : "PACKAGE" doesn't exist in the -schema.sql files 2022-12-27 11:00:36 [WARN] The object type : "TYPE" doesn't exist in the -schema.sql files 2022-12-27 11:00:36 [WARN] The object type : "PACKAGE_BODY" doesn't exist in the -schema.sql files 2022-12-27 11:00:36 [WARN] The object type : "TYPE_BODY" doesn't exist in the -schema.sql files 2022-12-27 11:00:36 [WARN] The object type : "SYNONYM" doesn't exist in the -schema.sql files 2022-12-27 11:00:36 [WARN] The object type : "PUBLIC_SYNONYM" doesn't exist in the -schema.sql files 2022-12-27 11:00:36 [WARN] The object type : "FILE" doesn't exist in the -schema.sql files 2022-12-27 11:00:37 [INFO] Close connection count: 1 of the DataSource. Key: xxx.xxx.xxx.xxx_11532_332378361_test_data 2022-12-27 11:00:37 [INFO] Shutdown task context finished 2022-12-27 11:00:37 [INFO] ---------- Finished Tasks: 2 Running Tasks: 0 Progress: 100.00% ---------- 2022-12-27 11:00:37 [INFO] All Load Tasks Finished: ---------------------------------------------------------------------------------------------------------------------------- No.# | Type | Name | Count | Status ---------------------------------------------------------------------------------------------------------------------------- 1 | TABLE | TEST_TBL2 | 1 -> 1 | SUCCESS 2 | TABLE | TEST_TBL1 | 1 -> 1 | SUCCESS ---------------------------------------------------------------------------------------------------------------------------- Total Count: 2 End Time: 2022-12-27 11:00:37 2022-12-27 11:00:37 [INFO] Load schema finished. Total Elapsed: 1.073 s 2022-12-27 11:00:37 [INFO] System exit 0Check the imported table schema.
obclient [test_data]> SHOW CREATE TABLE test_tbl1\G *************************** 1. row *************************** Table: test_tbl1 Create Table: CREATE TABLE `test_tbl1` ( `COL1` int(11) NOT NULL, `COL2` varchar(20) DEFAULT NULL, `COL3` int(11) DEFAULT NULL, PRIMARY KEY (`COL1`) ) DEFAULT CHARSET = utf8mb4 ROW_FORMAT = DYNAMIC COMPRESSION = 'zstd_1.3.8' REPLICA_NUM = 3 BLOCK_SIZE = 16384 USE_BLOOM_FILTER = FALSE TABLET_SIZE = 134217728 PCTFREE = 0 1 row in set
Migrate table data
This section uses CSV data files as an example to describe how to use obdumper and obloader to migrate table data.
Export CSV data files
File format definition: A CSV data file has the .csv file name extension. CSV is a file format that uses commas (,) as the separator. CSV data files store tabular data in the form of plain text. You can open them by using a text editor or Excel.
Scenario: Export all supported object data from the OBDUMPER_USER01 schema of the oracle001 tenant in the test4000 cluster to the /home/admin/test_migrate_data/csv_data directory in the CSV format.
The sample statement is as follows:
[xxx@xxx /ob-loader-dumper-4.0.0-RELEASE/bin] $./obdumper -h xxx.xxx.xxx.xxx -P 2883 -u obdumper_user01 -p ****** -c test4000 -t oracle001 -D obdumper_user01 --sys-user root --sys-password ****** --csv --table '*' -f /home/admin/test_migrate_data/csv_data 2022-12-27 10:02:20 [INFO] Parsed args: [--csv] true [--file-path] /home/admin/test_migrate_data/csv_data [--host] xxx.xxx.xxx.xxx [--port] 2883 [--user] obdumper_user01 [--tenant] oracle001 [--cluster] test4000 [--password] ****** [--database] obdumper_user01 [--sys-user] root [--sys-password] ****** [--table] [*] 2022-12-27 10:02:20 [INFO] Load jdbc driver class: "com.oceanbase.jdbc.Driver" finished 2022-12-27 10:02:21 [INFO] The manifest file: "/home/admin/test_migrate_data/csv_data/data/MANIFEST.bin" has been saved 2022-12-27 10:02:21 [INFO] Query column metadata for the table: "TEST_TBL1" finished 2022-12-27 10:02:21 [INFO] Query column metadata for the table: "TEST_TBL2" finished 2022-12-27 10:02:21 [INFO] Query partition names for table: "TEST_TBL1" success. (Non-partitioned) 2022-12-27 10:02:21 [INFO] Query partition names for table: "TEST_TBL2" success. (Non-partitioned) 2022-12-27 10:02:21 [INFO] Query primary key for table: "TEST_TBL1" success. Elapsed: 160.8 ms 2022-12-27 10:02:21 [INFO] Query primary key for table: "TEST_TBL2" success. Elapsed: 160.8 ms 2022-12-27 10:02:21 [INFO] Query table entry for table: "TEST_TBL2" success. Remain: 0. Elapsed: 4.120 ms 2022-12-27 10:02:21 [INFO] Query table entry for table: "TEST_TBL1" success. Remain: 0. Elapsed: 4.203 ms 2022-12-27 10:02:21 [INFO] Query all table entries success. Total: 2. Elapsed: 208.0 ms 2022-12-27 10:02:21 [INFO] ....Splitting rows for non-partitioned table: "TEST_TBL2" success. Batch: 1 2022-12-27 10:02:21 [INFO] ....Splitting rows for non-partitioned table: "TEST_TBL1" success. Batch: 1 2022-12-27 10:02:21 [INFO] ....Splitting rows for non-partitioned table: "TEST_TBL2" success. Batch: 2 2022-12-27 10:02:21 [INFO] ....Splitting rows for non-partitioned table: "TEST_TBL1" success. Batch: 2 2022-12-27 10:02:21 [INFO] Split rows for non-partitioned table(with primary key): "TEST_TBL2" success. Ranges: 2. Elapsed: 48.60 ms 2022-12-27 10:02:21 [INFO] Split rows for non-partitioned table(with primary key): "TEST_TBL1" success. Ranges: 2. Elapsed: 48.60 ms 2022-12-27 10:02:21 [INFO] Generate 4 dump tasks finished. Total Elapsed: 62.72 ms 2022-12-27 10:02:21 [INFO] Start 128 record dump threads for 4 dump tasks finished 2022-12-27 10:02:21 [INFO] Dump 5 rows OBDUMPER_USER01.TEST_TBL2 to "/home/admin/test_migrate_data/csv_data/data/OBDUMPER_USER01/TABLE/TEST_TBL2.1.*.csv" finished 2022-12-27 10:02:21 [INFO] Dump 5 rows OBDUMPER_USER01.TEST_TBL1 to "/home/admin/test_migrate_data/csv_data/data/OBDUMPER_USER01/TABLE/TEST_TBL1.1.*.csv" finished 2022-12-27 10:02:22 [INFO] Close connection count: 12 of the DataSource. Key: xxx.xxx.xxx.xxx_11532_840547833_obdumper_user01 2022-12-27 10:02:22 [INFO] Shutdown task context finished 2022-12-27 10:02:22 [INFO] ---------- Finished Tasks: 4 Running Tasks: 0 Progress: 100.00% ---------- 2022-12-27 10:02:22 [INFO] All Dump Tasks Finished: ---------------------------------------------------------------------------------------------------------------------------- No.# | Type | Name | Count | Status ---------------------------------------------------------------------------------------------------------------------------- 1 | TABLE | TEST_TBL2 | 5 | SUCCESS 2 | TABLE | TEST_TBL1 | 5 | SUCCESS ---------------------------------------------------------------------------------------------------------------------------- Total Count: 10 End Time: 2022-12-27 10:02:22 2022-12-27 10:02:22 [INFO] Unnecessary to merge the data files. As --file-name is missing 2022-12-27 10:02:22 [INFO] Dump record finished. Total Elapsed: 1.356 s 2022-12-27 10:02:22 [INFO] Unnecessary to upload the data files to the remote cloud storage service 2022-12-27 10:02:22 [INFO] System exit 0The directory structure exported is as follows:
[xxx@xxx /ob-loader-dumper-4.0.0-RELEASE/bin] $tree /home/admin/test_migrate_data/csv_data /home/admin/test_migrate_data/csv_data ├── data │ ├── CHECKPOINT.bin │ ├── MANIFEST.bin │ └── OBDUMPER_USER01 │ └── TABLE │ ├── TEST_TBL1.1.0.csv │ └── TEST_TBL2.1.0.csv └── logs ├── ob-loader-dumper.error ├── ob-loader-dumper.info └── ob-loader-dumper.warn 4 directories, 7 filesView the exported files.
[xxx@xxx /ob-loader-dumper-4.0.0-RELEASE/bin] $cat /home/admin/test_migrate_data/csv_data/data/OBDUMPER_USER01/TABLE/TEST_TBL1.1.0.csv 'COL1','COL2','COL3' 1,'China',86 2,'Taiwan',886 3,'Hong Kong',852 4,'Macao',853 5,'North Korea',850
Import CSV data files
Scenario: Export all supported CSV data files from the OBDUMPER_USER01 schema of the oracle001 tenant under the test4000 cluster to the /home/admin/test_migrate_data/csv_data directory, and import them into the test_data schema of the mysql001 tenant under the test4000 cluster.
Notice
- The directory where the
OBDUMPER_USER01schema in theoracle001tenant needs to be exported is/home/admin/test_migrate_data/csv_data/data/. The nameOBDUMPER_USER01in that directory should be changed totest_data. - The table names in the exported CSV file names should be changed to lowercase.
For example:
[root@xxx /home/admin/test_migrate_data/csv_data/data]# mv OBDUMPER_USER01 test_data
[root@xxx /home/admin/test_migrate_data/csv_data/data]# cd test_data/TABLE
[root@xxx /home/admin/test_migrate_data/csv_data/data/test_data/TABLE]# ls
TEST_TBL1.1.0.csv TEST_TBL2.1.0.csv
[root@xxx /home/admin/test_migrate_data/csv_data/data/test_data/TABLE]# mv TEST_TBL1.1.0.csv test_tbl1.1.0.csv
[root@xxx /home/admin/test_migrate_data/csv_data/data/test_data/TABLE]# mv TEST_TBL2.1.0.csv test_tbl2.1.0.csv
The sample statement is as follows:
[xxx@xxx /ob-loader-dumper-4.0.0-RELEASE/bin] $./obloader -h xxx.xxx.xxx.xxx -P 2883 -u obloader_user01 -p ****** --sys-user root --sys-password ****** -c test4000 -t mysql001 -D test_data --csv --table '*' -f /home/admin/test_migrate_data/csv_data 2022-12-27 11:17:21 [INFO] Parsed args: [--csv] true [--file-path] /home/admin/test_migrate_data/csv_data [--host] xxx.xxx.xxx.xxx [--port] 2883 [--user] obloader_user01 [--tenant] mysql001 [--cluster] test4000 [--password] ****** [--database] test_data [--sys-user] root [--sys-password] ****** [--table] [*] 2022-12-27 11:17:22 [INFO] Load jdbc driver class: "com.oceanbase.jdbc.Driver" finished 2022-12-27 11:17:22 [INFO] The manifest file: "/home/admin/test_migrate_data/csv_data/data/MANIFEST.bin" has been saved 2022-12-27 11:17:22 [INFO] Query column metadata for the table: "test_tbl1" finished 2022-12-27 11:17:22 [INFO] Query column metadata for the table: "test_tbl2" finished 2022-12-27 11:17:22 [INFO] File: "/home/admin/test_migrate_data/csv_data/data/CHECKPOINT.bin" is unmatched on the suffix[.csv], ignore it 2022-12-27 11:17:22 [INFO] File: "/home/admin/test_migrate_data/csv_data/data/MANIFEST.bin" is unmatched on the suffix[.csv], ignore it 2022-12-27 11:17:22 [INFO] Binding table: "test_tbl2" to the file: "/home/admin/test_migrate_data/csv_data/data/test_data/TABLE/test_tbl2.1.0.csv" finished 2022-12-27 11:17:22 [INFO] Binding table: "test_tbl1" to the file: "/home/admin/test_migrate_data/csv_data/data/test_data/TABLE/test_tbl1.1.0.csv" finished 2022-12-27 11:17:22 [INFO] Splitted 2 csv subfiles by 64.0 MB. Elapsed: 21.68 ms 2022-12-27 11:17:22 [INFO] Generate 2 subfiles finished 2022-12-27 11:17:22 [INFO] Ignore to clean any tables as --truncate-table or --delete-from-table is not specified 2022-12-27 11:17:22 [INFO] Query table entry and primary key for table: "test_tbl1" finished. Remain: 1 2022-12-27 11:17:22 [INFO] Query table entry and primary key for table: "test_tbl2" finished. Remain: 0 2022-12-27 11:17:22 [INFO] Query the leader location of "test_tbl1" finished. Remain: 0 2022-12-27 11:17:22 [INFO] Query the leader location of "test_tbl2" finished. Remain: 0 2022-12-27 11:17:22 [INFO] Calculate leader: xxx.xxx.xxx.xxx:2881 of table: "test_tbl1", part: 0. Remain: 1 2022-12-27 11:17:22 [INFO] Calculate leader: xxx.xxx.xxx.xxx:2881 of table: "test_tbl2", part: 0. Remain: 0 2022-12-27 11:17:22 [INFO] Waiting to refresh observer load status ...... 2022-12-27 11:17:22 [INFO] Refresh observer load status success. Table: "test_tbl1". Remain: 1 2022-12-27 11:17:22 [INFO] Refresh observer load status success. Table: "test_tbl2". Remain: 0 2022-12-27 11:17:22 [INFO] Refresh observer load status finished. Elapsed: 29.40 ms 2022-12-27 11:17:22 [INFO] Use c.l.d.PhasedBackoffWaitStrategy as available cpu(s) is 64 2022-12-27 11:17:22 [INFO] Create 4096 slots for ring buffer finished. [xxx.xxx.xxx.xxx:2881] 2022-12-27 11:17:22 [INFO] Start 128 database writer threads finished. [xxx.xxx.xxx.xxx:2881] 2022-12-27 11:17:22 [INFO] Start 128 csv file reader threads successed 2022-12-27 11:17:22 [INFO] File: "/home/admin/test_migrate_data/csv_data/data/test_data/TABLE/test_tbl2.1.0.csv" has been parsed finished 2022-12-27 11:17:22 [INFO] File: "/home/admin/test_migrate_data/csv_data/data/test_data/TABLE/test_tbl1.1.0.csv" has been parsed finished 2022-12-27 11:17:24 [INFO] Wait for the all the workers to drain of published events then halt the workers 2022-12-27 11:17:24 [INFO] Close connection count: 36 of the DataSource. Key: xxx.xxx.xxx.xxx_11532_332378361_test_data 2022-12-27 11:17:24 [INFO] Shutdown task context finished 2022-12-27 11:17:24 [INFO] ---------- Finished Tasks: 2 Running Tasks: 0 Progress: 100.00% ---------- 2022-12-27 11:17:24 [INFO] All Load Tasks Finished: ---------------------------------------------------------------------------------------------------------------------------- No.# | Type | Name | Count | Status ---------------------------------------------------------------------------------------------------------------------------- 1 | TABLE | test_tbl1 | 5 -> 5 | SUCCESS 2 | TABLE | test_tbl2 | 5 -> 5 | SUCCESS ---------------------------------------------------------------------------------------------------------------------------- Total Count: 10 End Time: 2022-12-27 11:17:24 2022-12-27 11:17:24 [INFO] Load record finished. Total Elapsed: 1.950 s 2022-12-27 11:17:24 [INFO] System exit 0Check the imported table data.
obclient [test_data]> SELECT * FROM test_tbl1; +------+-------------+------+ | COL1 | COL2 | COL3 | +------+-------------+------+ | 1 | China | 86 | | 2 | Taiwan | 886 | | 3 | Hong Kong | 852 | | 4 | Macao | 853 | | 5 | North Korea | 850 | +------+-------------+------+ 5 rows in set