Add a Kafka data source

You must configure data sources before you create a data synchronization project. Before you start a data synchronization project, you must add a Kafka instance as the destination data source. This topic describes how to add a Kafka data source in OceanBase Migration Service (OMS) Community Edition.

Limits

OMS Community Edition allows you to add a Kafka instance only as the destination for data synchronization.

Prerequisites

You have obtained the endpoint of the self-managed Kafka instance.

Background

Data security is a major concern throughout service connectivity verification, link creation, and data transmission. Based on the security system provided by the Kafka service, OMS Community Edition can meet most security requirements in data encryption and user authentication.

OMS Community Edition supports the following Kafka authentication methods:

• GSS-API GSS-API is short for Generic Security Services Application Program Interface. GSS-API is a framework that provides general security services to callers. It supports the Kerberos protocol.

• PLAIN PLAIN authentication is simple and does not support dynamic changes of users. In this authentication mode, usernames and passwords are configured in plaintext, resulting in low security.

• SCRAM-SHA-256 Salted Challenge Response Authentication Mechanism (SCRAM) authenticates users by username and password. SCRAM-SHA-256 can be used together with Transport Layer Security (TLS) for security authentication.

  In this authentication method, users can be dynamically changed, and the user data is stored in Zookeeper. Before a broker is started, a user for communication between the broker and Zookeeper must be created. However, usernames and passwords are configured in plaintext in this authentication method.

• SCRAM-SHA-512 SCRAM-SHA-512 can be used together with TLS for security authentication.

Procedure

1. Log on to the console of OMS Community Edition.

2. In the left-side navigation pane, click Data Source Management.

3. On the Data Sources page, click Add Data Source in the upper-right corner.

4. In the Add Data Source dialog box, select Kafka for Data Source Type.

5. Specify the following parameters.

   Parameter	Description
   Data Source Identifier	We recommend that you set it to a combination of Chinese characters, digits, and letters. It must not contain any spaces and cannot exceed 32 characters in length.
   Region	Select the region where the data source resides from the drop-down list. The region is the value that you set for the cm_region parameter when you deploy OMS Community Edition. Notice: This parameter is displayed only when multiple regions are available. Make sure that the mappings between the data source and the region are consistent. Otherwise, the migration and synchronization performance can be poor.
   Kafka Endpoint	The endpoint of the self-managed Kafka cluster, which is in the IP address:port number format. You can configure multiple nodes for the Kafka data source. Separate the broker addresses with commas (,), for example, xxx.xxx.xxx.1:2883,xxx.xxx.xxx.2:2883.
   Enable SSL	Choose whether to enable SSL based on the business needs. To enable SSL, click Upload File and upload an SSL certificate suffixed with .jks.
   Enable Authentication	Choose whether to enable authentication based on business needs. Kafka provides data encryption and multiple identity authentication mechanisms to ensure the security of user data and services.
   Authentication Method	If authentication is enabled, you must specify the authentication method. OMS Community Edition supports the following authentication methods: GSS-API, PLAIN, SCRAM-SHA-256, and SCRAM-SHA-512.
   KDC Server Address	The IP address or domain name of the Kerberos Key Distribution Center (KDC) server. This parameter is displayed only when you set Authentication Method to GSS-API.
   User Entity	Enter the username. This parameter is displayed only when you set Authentication Method to GSS-API.
   SSL Certificate	Click Upload File and upload a key file suffixed with .keytab. This parameter is displayed only when you set Authentication Method to GSS-API.
   Username	The name of the MySQL database user for data synchronization. This parameter is not displayed when you set Authentication Method to GSS-API.
   Password	The password of the MySQL database user for data synchronization. This parameter is not displayed when you set Authentication Method to GSS-API.
   Remarks (Optional)	Additional information about the data source.

6. Click Test Connection to test the network connection between OMS Community Edition and the self-managed Kafka instance.

7. After the connection test is passed, click Add.