This topic describes how to use OceanBase Migration Service (OMS) Community Edition to migrate data from an ElasticSearch database to OceanBase Database Community Edition.
Limitations
Limitations on the source database
Do not perform DDL operations that change database or schema structures during full data migration. Otherwise, the data migration task may be interrupted.
Only full data migration is supported when you migrate data from an ElasticSearch database to an OceanBase Community Edition database.
Data source identifiers and user accounts must be unique in the OMS system.
OMS Community Edition only supports migrating databases, tables, and column objects with ASCII-compliant names that do not contain special characters (|"'`()=;/& and spaces).
We recommend that you do not migrate more than 1,000 tables at a time to avoid impacting the migration task.
Data type mappings
ElasticSearch |
OceanBase Database Community Edition |
|---|---|
| string | varchar |
| text | varchar |
| keyword | varchar |
| short | integer |
| integer | integer |
| long | bigint |
| float | float |
| double | double |
| half_float | float |
| scaled_float | float |
| date | timestamp |
| boolean | tinyint |
| binary | binary |
| array | array,varchar |
| object | text |
| nested | text |
| geo_point | point |
| geo_shape | geometry |
| ip | varchar |
| dense_vector | vector |
Procedure
Create a data migration task.
Log in to the OMS Community Edition console.
In the left-side navigation pane, click Data Migration.
On the Migration Tasks page, click New Task in the upper-right corner.
On the Select Source and Destination page, configure the parameters.
ParameterDescriptionTask Name We recommend that you use Chinese, English, and numbers. The task name cannot contain spaces and must be 64 characters or less. Tag Click the field and select a tag from the drop-down list. You can also click Manage Tags to create, modify, or delete tags. For more information, see Manage data migration tasks by using tags. Source If you have created an ElasticSearch data source, select one from the drop-down list. If not, click New Data Source in the drop-down list. The right-side dialog box allows you to create a data source. For more information, see Create an ElasticSearch data source. Destination If you have created an OceanBase Community Edition data source, select one from the drop-down list. If not, click New Data Source in the drop-down list. The right-side dialog box allows you to create a data source. For more information, see Create an OceanBase-CE data source. Click Next.
On the Select Migration Type page, select Full Migration.
After the full migration starts, OMS Community Edition migrates the existing data in the source table to the corresponding table in the destination database.
Click Next. On the Select Migration Objects page, select the migration objects and scope.
You can select migration objects by using the Specify Objects and Match Rules options. This topic describes the procedure for selecting migration objects by using the Specify Objects option. For information about how to configure matching rules, see Configure matching rules for migration objects.
Notice
The name of a table to be migrated, as well as the names of columns in the table, must not contain Chinese characters.
If the name of a database or table contains the characters "$$", the data migration task cannot be created.
In the Select Migration Objects section, select Specify Objects.
In the Specify Migration Scope section, select the source objects to migrate in the Source Object(s) list.
Click > to add the selected objects to the Target Object(s) list.
When you migrate data from an ElasticSearch database to an OceanBase Community Edition database, OMS Community Edition allows you to import objects by using text, rename objects, and remove one or all migration objects.
OperationProcedureImport objects - In the right-side list of the Specify Migration Scope section, click Import Objects in the upper-right corner.
- In the dialog box that appears, click OK.
Notice:
The import operation will overwrite the previous selection. Proceed with caution. - In the Import Migration Objects dialog box, import the objects to be migrated.
You can rename the database and tables by importing a CSV file. For more information, see Download and import settings of migration objects. - Click Validate.
- After the validation succeeds, click OK.
Rename objects OMS Community Edition allows you to rename migration objects. For more information, see Rename database and table objects. Remove objects OMS Community Edition allows you to remove one or more selected objects from the destination database during data mapping. - Remove a single migration object
In the right-side list of the Specify Migration Scope section, hover the pointer over the target object and click Remove in the drop-down list. - Remove all migration objects
In the right-side list of the Specify Migration Scope section, click Remove All in the upper-right corner. In the dialog box that appears, click OK to remove all migration objects.
Click Next. On the Migration Type page, configure the parameters.
Notice
The parameters described below will only be displayed after you select Full Migration on the Select Migration Type page.
ParameterDescriptionConcurrency Valid values: Stable, Normal, Fast, and Custom. The performance of a full migration task varies with the concurrency speed. The resources required for a full migration task also vary with the concurrency speed. If you set the concurrency speed to Custom, you can set the Read Concurrency, Write Concurrency, and JVM Memory parameters based on your business requirements. Handling strategy when the destination object contains records Valid values: Ignore and Stop Migration: - If you set the value to Ignore, OMS Community Edition will log the conflicting data and retain the original data when original data conflicts with data to be written to the table during full verification.
Notice
If you set the value to Ignore, full verification will use the IN mode to fetch data, which cannot detect cases where the destination contains data not present in the source, and the verification performance will be degraded to some extent.
- If you set the value to Stop Migration, OMS Community Edition will report an error and stop migration when the destination contains data. Make sure that the destination data is properly handled before you continue the migration.
Notice
If you click Resume after an error occurs, OMS Community Edition will ignore this setting and continue to migrate table data. Proceed with caution.
Write Method Valid values: SQL, which means to write data to tables by using INSERTorREPLACEstatements, and Direct Load, which means to write data to tables by using direct load. At present, data of vector types cannot be written to tables by using the direct load method.To view or modify the parameters of the full migration component, click Full Migration > Configuration Details in the upper-right corner. For more information about the parameters, see Component parameters.
- If you set the value to Ignore, OMS Community Edition will log the conflicting data and retain the original data when original data conflicts with data to be written to the table during full verification.