Deploy OceanBase Database in a production environment by using the command line

This topic describes how to deploy OceanBase Database in a production environment by using obd in a CLI mode.

Terms

• Central control server

  A server that runs obd and is used to manage the OceanBase cluster and store the installation packages and configuration information of the OceanBase cluster.

• Target server

  A server that runs OceanBase Database.

• obd

  OceanBase Deployer is a tool used to install and deploy OceanBase Database. It is abbreviated as obd. For more information, see obd documentation.

• ODP

  OceanBase Database Proxy, abbreviated as ODP, is a dedicated proxy server for OceanBase Database. It is also known as OBProxy. For more information, see ODP documentation.

• OCP

  OceanBase Cloud Platform, abbreviated as OCP. For more information, see OCP documentation.

• OCP Express

  OCP Express is a web-based management tool for OceanBase Database 4.x. Integrated with an OceanBase cluster, OCP Express allows you to view key performance metrics of the cluster and perform basic database management operations on the cluster. For more information, see OceanBase Cloud Platform Express (OCP Express).

Prerequisites

Before you deploy OceanBase Database, make sure that:

• Your server meets the software and hardware requirements. For more information, see Hardware and software requirements.

• In a production environment, you have performed environment and configuration checks. For more information, see Configure before deployment.

• You have installed obd on the CVM. We recommend that you install the latest version. For more information, see Install and configure obd.

• You have installed the OBClient. For more information, see OBClient documentation.

Deployment modes

This topic describes the deployment of three replicas of OceanBase Database. You can choose a suitable deployment solution based on your business requirements. The following table describes the deployment of four servers for OceanBase Database in this example.

Procedure

Step 1: (Optional) Download and install the all-in-one package

OceanBase Database provides a unified all-in-one package from V4.0.0. You can use this package to install components such as obd, OceanBase Database, ODP, and OBAgent at one time.

You can also choose to download and install components from OceanBase Download Center as needed or specify the versions of the components.

Install obd online

If your server can connect to the Internet, run the following command to install components online.

[admin@test001 ~]$ bash -c "$(curl -s https://obbusiness-private.oss-cn-shanghai.aliyuncs.com/download-center/opensource/oceanbase-all-in-one/installer.sh)"
[admin@test001 ~]$ source ~/.oceanbase-all-in-one/bin/env.sh

Install obd offline

If your server cannot connect to the Internet, follow these steps to install components offline.

1. Download the latest all-in-one package from OceanBase Download Center and copy it to any directory on the master server.

2. In the directory where the package is located, run the following command to decompress and install the package.

   [admin@test001 ~]$ tar -xzf oceanbase-all-in-one-*.tar.gz
   [admin@test001 ~]$ cd oceanbase-all-in-one/bin/
   [admin@test001 bin]$ ./install.sh
   [admin@test001 bin]$ source ~/.oceanbase-all-in-one/bin/env.sh
   

Step 2: Configure obd

If you deploy the OceanBase cluster offline, you can refer to Step 1 to download and install the all-in-one package on the master server. The components in the all-in-one package have been tested for compatibility. This package is therefore recommended.

If you need to deploy components of specific versions offline, you can download the installation package of the required version from OceanBase Download Center. Copy the installation package to any directory on the master server and follow steps 1 to 3 below to upload the package to the local image repository of obd.

1. Disable remote repositories

   [admin@test001 rpm]$ obd mirror disable remote
   

   Note

   By default, remote repositories are disabled after you install the all-in-one package. You can run the obd mirror list command to confirm this. If the Enabled value of Type=remote is False, remote image sources are disabled.

2. Add the installation package to the local image repository

   [admin@test001 rpm]$ obd mirror clone *.rpm
   

3. View the list of installation packages in the local image repository

   [admin@test001 rpm]$ obd mirror list local
   

4. Select a configuration file

   If obd was installed by direct download on your server, you can view sample configuration files in the /usr/obd/example directory.

   If obd was installed by decompressing the all-in-one package on your server, you can view sample configuration files in the ~/.oceanbase-all-in-one/conf directory. Select a configuration file based on your resource conditions.

   Note

   Configuration files are divided into mini and professional development modes. The two modes have basically the same parameters, with slight differences in specifications. You can select a configuration file based on your resource conditions.

   • Mini development mode: This mode is suitable for personal devices with at least 8 GB of memory. The names of the configuration files contain the mini or min identifier.

   • Professional development mode: This mode is suitable for high-configured ECS instances or physical servers with at least 16 CPU cores and 64 GB of memory.

   If you plan to deploy the OceanBase cluster on a single server, use the configuration file for single-server deployment.

   • Local single-server deployment sample configuration files: mini-local-example.yaml and local-example.yaml

   • Single-server deployment sample configuration files: mini-single-example.yaml and single-example.yaml

   • Single-server deployment with ODP sample configuration files: mini-single-with-obproxy-example.yaml and single-with-obproxy-example.yaml

   If you plan to deploy the OceanBase cluster across multiple servers, use the configuration file for multi-server deployment.

   • Multi-server deployment sample configuration files: mini-distributed-example.yaml and distributed-example.yaml

   • Multi-server deployment with ODP sample configuration files: mini-distributed-with-obproxy-example.yaml and distributed-with-obproxy-example.yaml

   • Multi-server deployment with ODP and OCP Express sample configuration files: default-components.yaml and default-components-min.yaml

   • Complete multi-server deployment sample configuration files: all-components-min.yaml and all-components.yaml

   Note

   If you plan to deploy OCP Express or have deployed OCP, you do not need to deploy Prometheus and Grafana.

5. Modify the configuration file.

   The following example shows how to modify the configuration file when you use the default-components.yaml file to deploy a distributed OceanBase cluster.

   Note

   You must modify the following parameters based on your environment.

   1. Modify the username and password.

      ## Only need to configure when remote login is required
      user:
        username: admin
      #   password: your password if need
        key_file: /home/admin/.ssh/id_rsa
      #   port: your ssh port, default 22
      #   timeout: ssh connection timeout (second), default 30
      

      username specifies the username for logging in to the target server. Make sure that you have write permission to home_path for the username. password and key_file are used to verify the user. Generally, you only need to fill in one of them.

      Notice

      After you configure the key path, if your key does not require a password, comment out or delete the password parameter. This is to prevent the password parameter from being mistakenly used as the password for logging in to the key, which will cause a verification failure.

   2. Modify the IP address, ports, and related directories, and configure the memory and password parameters.

      oceanbase-ce:
        servers:
          # Please don't use hostname, only IP can be supported
          - name: server1
            ip: 10.10.10.1
          - name: server2
            ip: 10.10.10.2
          - name: server3
            ip: 10.10.10.3
        global:
          devname: eth0
          cluster_id: 1
          # please set memory limit to a suitable value which is matching resource. 
          memory_limit: 64G # The maximum running memory for an observer
          system_memory: 30G # The reserved system memory. system_memory is reserved for general tenants.
          datafile_size: 192G # Size of the data file. 
          datafile_next: 200G
          datafile_maxsize: 1T
          log_disk_size: 192G # The size of disk space used by the clog files.
          enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true.
          max_syslog_file_count: 4 # The maximum number of reserved log files before enabling auto recycling. The default value is 0.
          # observer cluster name, consistent with obproxy's cluster_name
          appname: obdemo
          ocp_meta_tenant: # The config for ocp express meta tenant
            tenant_name: ocp
            max_cpu: 1
            memory_size: 2G
            log_disk_size: 7680M
          mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started.
          rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started.
          obshell_port: 2886 # Operation and maintenance port for OceanBase Database.
          # The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
          home_path: /home/admin/observer
          # The directory for data storage. The default value is $home_path/store.
          data_dir: /data
          # The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
          redo_dir: /redo
          root_password: ****** # root user password, can be empty
          proxyro_password: ******** # proxyro user pasword, consistent with obproxy's observer_sys_password, can be empty
        server1:
          zone: zone1
        server2:
          zone: zone2
        server3:
          zone: zone3
      

      In the case of inconsistent configurations on the servers, you can move the related configurations to the corresponding server and set the priority of the server's configurations higher than that of global. For example, the following example shows the case where the ports on two servers are different:

      server2:
        mysql_port: 3881
        rpc_port: 3882
        zone: zone2
      server3:
        mysql_port: 2881
        rpc_port: 2882
        zone: zone3 
      

      Parameter	Required	Default value	Description
      servers	Required	N/A	You must specify each server by using - name: server name (newline)ip: IP address. You can specify multiple servers by repeating the parameter, but the server names must be unique. If the server IP addresses are unique, you can also specify the servers by using - <ip> (newline) - <ip>. In this case, the - <ip> format is equivalent to - name: server IP address (newline)ip: server IP address.
      devname	Optional	N/A	The network interface controller (NIC) corresponding to the IP address specified in servers. You can run the ip addr command to view the correspondence between the IP address and the NIC.
      memory_limit	Optional	0	The maximum memory, in bytes, that the observer process can use from the environment. If this parameter is not specified, the value of memory_limit_percentage takes effect. For more information, see memory_limit and memory_limit_percentage.
      system_memory	Optional	0M	The reserved system memory. The value of this parameter consumes the memory specified by memory_limit. If this parameter is not specified, OceanBase Database adapts to the memory.
      datafile_size	Optional	0	The size of the data file (block_file) for the corresponding node. If this parameter is not specified, the value of datafile_disk_percentage takes effect. For more information, see datafile_size and datafile_disk_percentage.
      datafile_next	Optional	0	The growth step of disk space. It is used to set auto expansion. If this parameter is not specified, you can refer to Configure dynamic expansion of disk data files to enable auto expansion.
      datafile_maxsize	Optional	0	The maximum available upper limit of disk space. It is used to set auto expansion. If this parameter is not specified, you can refer to Configure dynamic expansion of disk data files to enable auto expansion.
      log_disk_size	Optional	0	The size of the redo log disk. If this parameter is not specified, the value of log_disk_percentage takes effect. For more information, see log_disk_size and log_disk_percentage.
      enable_syslog_wf	Optional	true	Specifies whether to print system logs with a level of WARN or higher to a separate log file.
      max_syslog_file_count	Optional	0	The number of log files that can be accommodated before the system recycles log files. A value of 0 indicates that automatic recycling is not performed.
      appname	Optional	obcluster	The name of the OceanBase cluster.
      ocp_meta_tenant	Optional	tenant_name: ocp max_cpu: 1 memory_size: 2147483648	The specifications of the meta tenant required for OCP Express. The parameters specified in ocp_meta_tenant are passed as parameters when the tenant is created. Only several important parameters are listed in the preceding example. For more information about the parameters, see the parameters supported by the obd cluster tenant create command. For more information, see Cluster commands.
      mysql_port	Required	2881	The port number for the SQL service. The default value is 2881.
      rpc_port	Required	2882	The port number for remote access. It is the RPC communication port between the observer process and other node processes. The default value is 2882.
      obshell_port	Required	2886	The port number for OceanBase Database maintenance. The default value is 2886.
      home_path	Required	N/A	The installation path of OceanBase Database. We recommend that you specify this path under the admin user.
      data_dir	Optional	$home_path/store	The directory for storing data files such as SSTables. We recommend that you specify an independent disk for this directory.
      redo_dir	Optional	The same as data_dir	The directory for clogs. The default value is the same as that of data_dir. We recommend that you specify an independent disk for this directory.
      root_password	Optional	Empty by default in versions earlier than V2.1.0 Random string by default in V2.1.0 and later	The password of the root user of the sys tenant, which is the super administrator of the OceanBase cluster. We recommend that you set a complex password. If you use obd V2.1.0 or later, a random string is automatically generated if you do not specify this parameter.
      proxyro_password	Optional	Empty by default in versions earlier than V2.1.0 Random string by default in V2.1.0 and later	The password of the proxyro user of the sys tenant, which is used to connect to the OceanBase cluster. If you use obd V2.1.0 or later, a random string is automatically generated if you do not specify this parameter.

   3. Configure the obproxy-ce component and modify the IP address and home_path.

      obproxy-ce:
        depends:
          - oceanbase-ce
        servers:
          - 10.10.10.4
        global:
          listen_port: 2883 # External port. The default value is 2883.
          prometheus_listen_port: 2884 # The Prometheus port. The default value is 2884.
          home_path: /home/admin/obproxy
          enable_cluster_checkout: false
          skip_proxy_sys_private_check: true
          enable_strict_kernel_release: false
          obproxy_sys_password: ****** # obproxy sys user password, can be empty. When a depends exists, obd gets this value from the oceanbase-ce of the depends.
          observer_sys_password: ***** # proxyro user pasword, consistent with oceanbase-ce's proxyro_password, can be empty. When a depends exists, obd gets this value from the oceanbase-ce of the depends.
      

      Parameter	Required	Default value	Description
      servers	Required	N/A	You must specify each server by using - name: server name (newline)ip: IP address. You can specify multiple servers by repeating the parameter, but the server names must be unique. If the server IP addresses are unique, you can also specify the servers by using - <ip> (newline) - <ip>. In this case, the - <ip> format is equivalent to - name: server IP address (newline)ip: server IP address.
      listen_port	Required	2883	The listening port of ODP. The default value is 2883.
      prometheus_listen_port	Required	2884	The listening port of ODP for Prometheus. The default value is 2884.
      home_path	Required	N/A	The installation path of ODP. We recommend that you specify this path under the admin user.
      obproxy_sys_password	Optional	Empty by default in versions earlier than V2.1.0 Random string by default in V2.1.0 and later	The password of the root user of the proxysys tenant, which is the administrator of ODP. If you use obd V2.1.0 or later, a random string is automatically generated if you do not specify this parameter.
      observer_sys_password	Optional	Empty by default in versions earlier than V2.1.0 Random string by default in V2.1.0 and later	The password of the proxyro user of the sys tenant, which is used to connect to the OceanBase cluster. The value of this parameter must be the same as that of proxyro_password. If you use obd V2.1.0 or later, a random string is automatically generated if you do not specify this parameter.

   4. Modify the IP address and home_path for the obagent and ocp-express components.

      Note

      Starting from V2.0.0, obd allows you to deploy OCP Express. However, obd V2.0.0 supports only OCP Express V1.0.0, and OCP Express V1.0.1 and later versions can be deployed only by using obd V2.1.0 or later.

      obagent:
        depends:
          - oceanbase-ce
        servers:
          - name: server1
            # Please don't use hostname, only IP can be supported
            ip: 10.10.10.1
          - name: server2
            ip: 10.10.10.2
          - name: server3
            ip: 10.10.10.3
        global:
          home_path: /home/admin/obagent
      ocp-express:
        depends:
          - oceanbase-ce
          - obproxy-ce
          - obagent
        servers:
          - name: server1
            ip: 10.10.10.4
        global:
          # The working directory for prometheus. prometheus is started under this directory. This is a required field.
          home_path: /home/oceanbase/ocp-server
          memory_size: 1G # The memory size of ocp-express server.  The recommend value is 512MB + (expect node num + expect tenant num) * 60MB.
          admin_passwd: ********
          logging_file_total_size_cap: 10GB # The total log file size of ocp-express server
          # logging_file_max_history: 1 # The maximum of retention days the log archive log files to keep. The default value is unlimited
      

      Parameter	Required	Default value	Description
      servers	Required	N/A	You must specify each server by using - name: server name (newline)ip: IP address. You can specify multiple servers by repeating the parameter, but the server names must be unique. If the server IP addresses are unique, you can also specify the servers by using - <ip> (newline) - <ip>. In this case, the - <ip> format is equivalent to - name: server IP address (newline)ip: server IP address.
      home_path	Required	N/A	The installation path of the component. We recommend that you specify this path under the admin user.
      memory_size	Required	N/A	The size of memory, in bytes, for an OCP Express server. We recommend that you use the following formula to calculate the memory size: memory_size = 512MB + (expected number of nodes × expected number of tenants) × 60MB. The expected number of tenants must include the sys tenant and the OCP meta tenant.
      logging_file_total_size_cap	Required	1GB	The total size of log files. The default value is 1 GB. Note The unit of this parameter must be GB or MB. If you use G or M as the unit, an error occurs and OCP Express cannot be deployed.
      admin_passwd	Required	N/A	The password of the admin user for logging in to OCP Express. The password must be 8 to 32 characters in length and must contain at least two digits, two uppercase letters, two lowercase letters, and two special characters (~!@#%^&*_-+=|(){}[]:;,.?/). If you use obd V2.1.0 or later, a random string is automatically generated if you do not specify this parameter.

Step 3: Deploy an OceanBase cluster

1. Deploy the OceanBase cluster.

   [admin@test001 ~]$ obd cluster deploy obtest -c default-components.yaml
   

   If the target server is connected to the Internet, obd will check whether the installation package is deployed on the target server after you run the obd cluster deploy command. If not, obd will automatically obtain the installation package from the YUM source.

2. Start the OceanBase cluster.

   [admin@test001 ~]$ obd cluster start obtest
   

   If OCP Express is installed, the access address of OCP Express is displayed. On Alibaba Cloud or in other cloud environments, the installer may output an intranet IP address because it cannot obtain a public IP address. Make sure that you use the correct IP address.

3. Query the status of the OceanBase cluster.

   # Query the list of clusters managed by obd
   [admin@test001 ~]$ obd cluster list
   
   # Query the status of the obtest cluster
   [admin@test001 ~]$ obd cluster display obtest
   

4. (Optional) Modify the cluster settings.

   OceanBase Database has hundreds of parameters, and some parameters are coupled. We recommend that you do not modify the parameters in the sample configuration file before you become familiar with OceanBase Database. The following example shows how to modify a parameter and make the modification take effect.

   Run the obd cluster edit-config command to go to the parameter editing mode, and modify the cluster parameters.

   [admin@test001 ~]$ obd cluster edit-config obtest
   

   After you modify and save the parameters and exit, obd will prompt you how to make the modification take effect. Enter the following command after you save the modifications:

   Search param plugin and load ok
   Search param plugin and load ok
   Parameter check ok
   Save deploy "obtest" configuration
   Use `obd cluster reload obtest` to make changes take effect.
   

   Copy and execute the command provided by obd.

   [admin@test001 ~]$ obd cluster reload obtest
   

Step 4: Connect to the OceanBase cluster

In this example, OBClient is used to connect to the OceanBase cluster:

obclient -h<IP> -P<PORT> -u<user_name>@<tenant_name>#<cluster_name> -p -c -A
# example
obclient -h10.10.10.4 -P2883 -uroot@sys#obdemo -p -c -A

The parameters are described as follows:

• -h: specifies the IP address for connecting to the OceanBase Database. If you connect directly to the database, use the address of the OBServer node. If you connect to the database through an ODP, use the address of the ODP.

• -u: specifies the account for connecting to the OceanBase Database. The account format is one of the following: username@tenant name#cluster name, cluster name:tenant name:username, cluster name-tenant name-username, or cluster name.tenant name.username. The username of the administrator of a MySQL tenant is default root.

  Notice

  The cluster name used for connection is the value of appname in the configuration file, rather than the value of deploy name.

• -P: specifies the port for connecting to the OceanBase Database. If you connect directly to the database, use the value of mysql_port. If you connect to the database through an ODP, use the value of listen_port.

• -p: specifies the password for connecting to the OceanBase Database.

• -c: specifies that comments should not be ignored in the OBClient runtime environment.

  Note

  Hint, which is a special comment, is not affected by the -c option.

• -A: specifies that statistics should not be automatically retrieved when OBClient connects to the database.

For more information about how to connect to an OceanBase cluster, see Connect to OceanBase Database.

Step 5: Create a user tenant

After you deploy an OceanBase cluster, we recommend that you create a user tenant for business operations. The sys tenant is intended only for cluster management and is unsuitable for business scenarios.

You can create a user tenant by using one of the following methods:

• Method 1: Use obd to create a user tenant.

  obd cluster tenant create <deploy name> [-n <tenant name>] [flags]
  # example
  obd cluster tenant create test -n obmysql --max-cpu=2 --memory-size=2G --log-disk-size=3G --max-iops=10000 --iops-weight=2 --unit-num=1 --charset=utf8
  

  By default, this command creates a tenant based on all the remaining resources of the cluster. You can specify parameters to control the resources that the tenant uses. For more information about how to use this command, see the obd cluster tenant create topic in the obd documentation.

• Method 2: Create a user tenant on the command line. For more information, see Create a tenant.

References

Manage clusters

You can run the following commands to manage a cluster deployed by using obd. For more information, see Cluster commands.

# View the list of clusters
obd cluster list

# View the status of a cluster. Here is an example of viewing the status of the obtest cluster.
obd cluster display obtest

# Stop a running cluster. Here is an example of stopping the obtest cluster.
obd cluster stop obtest

# Destroy a deployed cluster. Here is an example of destroying the obtest cluster.
obd cluster destroy obtest

Uninstall all-in-one

You can follow the steps below to uninstall the installed all-in-one package.

1. Uninstall the installation package.

   [admin@test001 ~]$ cd oceanbase-all-in-one/bin/
   [admin@test001 bin]$ ./uninstall.sh
   

2. Delete the environment variables.

   [admin@test001 bin]$ vim ~/.bash_profile
   # Delete source /home/admin/.oceanbase-all-in-one/bin/env.sh from the file.
   

3. Apply the changes.

   After you delete the environment variables, you must log in to the terminal again or run the source command to apply the changes. Here is an example:

   [admin@test001 ~]$ source ~/.bash_profile