Integrate OceanBase Database with Linkis

Linkis is an open-source distributed computing middleware that offers unified management of computing resources, task scheduling, and execution services. Linkis creates an intermediary layer that allows different data processing applications (such as Apache Spark, Hadoop, and TensorFlow) to interact with data storage and computing resources.

Linkis V3.1.2 supports the OceanBase database. This article introduces how to create OceanBase database tasks in Linkis. OceanBase database is compatible with most of the features and syntax of MySQL 5.7. Therefore, you can use OceanBase Database as if it were MySQL.

Prerequisites

Before integrating OceanBase Database, make sure that you have installed and deployed Linkis. For more information, see Linkis Deployment Guide.

Submit tasks using shell

 sh ./bin/linkis-cli -engineType jdbc-4 -codeType jdbc -code "show tables" -submitUser hadoop -proxyUser hadoop -runtimeMap wds.linkis.jdbc.connect.url=jdbc:mysql://${ip}:${port}/${db_name} -runtimeMap wds.linkis.jdbc.driver=com.mysql.jdbc.Driver -runtimeMap wds.linkis.jdbc.username=${username} -runtimeMap wds.linkis.jdbc.password=${password}

Submit tasks using Linkis SDK

Linkis provides SDKs for Java and Scala to submit tasks to the Linkis server. For more information, see Java SDK Guide. For OceanBase tasks, you only need to modify the EngineConnType and CodeType parameters in the Demo :

Map<String, Object> labels = new HashMap<String, Object>();
labels.put(LabelKeyConstant.ENGINE_TYPE_KEY, "jdbc-4"); // required engineType Label
labels.put(LabelKeyConstant.USER_CREATOR_TYPE_KEY, "hadoop-IDE");// required execute user and creator
labels.put(LabelKeyConstant.CODE_TYPE_KEY, "jdbc"); // required codeType

Submit OceanBase tasks through the Linkis console

Add OceanBase Data Source

1. Log in to the Linkis console.

2. In the left navigation bar, click on Data Source.

3. In Data Source page, click New in the upper right corner.

4. In New Data Source form, select MySQL from Type.

5. Configure parameters.

   Parameter	Description
   name	Data source identifier
   Host	IP address of the OceanBase tenant
   Port	Port of the OceanBase tenant
   Username	Username of the OceanBase tenant
   Password	Password of the OceanBase tenant

6. Click Test Connection, Verify the network connection between Linkis and the database, as well as the validity of the username and password.

7. After the connection test is passed, click Finish.

8. Publish Data Source.

   In Data Source Page, click OceanBase version, in the edit version pop-up window, click Publish.

Submit OceanBase Tasks

Submit an OceanBase task by specifying the data source name.

Request URL: http://${gateway_url}:${port}/api/rest_j/v1/entrance/submit

Method: POST

Parameter:

{
    "executionContent": {
        "code": "show databases",
        "runType": "jdbc"
    },
    "params": {
        "variable": {},
        "configuration": {
            "startup": {},
            "runtime": {
                "wds.linkis.engine.runtime.datasource": "ob-test"
            }
        }
    },
    "labels": {
        "engineType": "jdbc-4"
    }
}

Response:

{
    "method": "/api/entrance/submit",
    "status": 0,
    "message": "OK",
    "data": {
        "taskID": 93,
        "execID": "exec_id018017linkis-cg-entrance000830fb1364:9104IDE_hadoop_jdbc_0"
    }
}

Reference

For more information, see following documents:

• Linkis integration with OceanBase
• Linkis datasource