Use SpringBatch to connect to OceanBase Database|OceanBase Database|docs|OceanBase Documentations

Applicability

SpringBatch is applicable to OceanBase Database in MySQL and Oracle modes. This topic provides only an example for the Oracle mode.

This topic describes how to use the SpringBatch framework and OceanBase Database to build an application that can perform basic operations such as creating tables, inserting data, and querying data.

Download the java-oceanbase-springbatch sample project

Prerequisites

You have installed OceanBase Database.
You have installed JDK 1.8 and Maven.
You have installed IntelliJ IDEA.

Note

The code in this topic is run in IntelliJ IDEA 2021.3.2 (Community Edition). You can also use other tools that you prefer to run the sample code.

Procedure

Note

The operations described in this topic are performed in a Windows environment. If you are using a different operating system or compiler, the operations may vary.

Obtain the connection string of OceanBase Database.
Import the java-oceanbase-springbatch project into IntelliJ IDEA.
Modify the database connection information in the java-oceanbase-springbatch project.
Run the java-oceanbase-springbatch project.

Step 1: Obtain the connection string of OceanBase Database

Contact the OceanBase Database deployment personnel or administrator to obtain the corresponding database connection string.
```
obclient -hxx.xx.xx.xx -P2883 -uroot@sys#cluster -p**** -A
```
Fill in the following URL with the information of the deployed OceanBase Database.

Note

The URL information is required in the application.properties file.
```
jdbc:oceanbase://host:port/schema_name?user=$user_name&password=$password&characterEncoding=utf-8
```
Parameter description:
- host: the IP address for connecting to OceanBase Database. For ODP connection, it is the IP address of an ODP. For direct connection, it is the IP address of an OBServer node.
- port: the port for connecting to OceanBase Database. For ODP connection, the default value is 2883, which can be customized when ODP is deployed. For direct connection, the default value is 2881, which can be customized when OceanBase Database is deployed.
- schema_name: the name of the schema to be accessed.
- user_name: the username, in the user@tenant#cluster name or username@SERVICE:service name format. If you use the user@tenant#cluster name format, the default tenant is sys and the administrator user is root. When you directly connect to the database, you can omit the cluster name. When you connect through ODP, you must specify the cluster name.
- password: the password of the account.
- characterEncoding: the character encoding.

For more information about URL parameters, see Database URL.

Step 2: Import the `java-oceanbase-springbatch` project into IntelliJ IDEA

Start IntelliJ IDEA and choose File > Open....
In the Open File or Project window that appears, select the project file and click OK.
IntelliJ IDEA automatically identifies various files in the project and displays the project structure, file list, module list, and dependency relationships in the Project tool window. The Project tool window is usually located on the left side of the IntelliJ IDEA interface and is open by default. If it is closed, you can click View > Tool Windows > Project in the menu bar or press the shortcut key Alt + 1 to reopen it.

Note

When you import a project by using IntelliJ IDEA, it automatically detects the pom.xml file in the project, downloads the required dependency libraries based on the described dependencies in the file, and adds them to the project.
View the project.

springbatch

Step 3: Modify the database connection information in the `java-oceanbase-springbatch` project

Modify the database connection information in the application.properties file based on the information obtained in Step 1: Obtain the connection string of OceanBase Database.

Here is an example:

The name of the database driver is com.mysql.cj.jdbc.Driver.
The IP address of the OBServer node is 10.10.10.1.
The access port is 2881.
The name of the schema to be accessed is test.
The tenant account is root@mysql001. mysql001 is a MySQL tenant created in OceanBase Database, and root is the username of the mysql001 tenant.
The password is ******.

Here is the sample code:

spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://10.10.10.1:2881/test?characterEncoding=utf-8
spring.datasource.username=root@mysql001
spring.datasource.password=******

spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update

spring.batch.job.enabled=false

logging.level.org.springframework=INFO
logging.level.com.example=DEBUG

Step 4: Run the `java-oceanbase-springbatch` project

Run the AddDescPeopleWriterTest.java file.
1. Find the AddDescPeopleWriterTest.java file in src > test > java in the project structure.
2. In the tool menu, choose Run > Run... > AddDescPeopleWriterTest.testWrite, or click the green triangle in the upper-right corner to run it.
3. View the log information and output results in the console of IntelliJ IDEA.
```
Data in the people_desc table:
PeopleDESC [name=John, age=25, desc=This is John with age 25]
PeopleDESC [name=Alice, age=30, desc=This is Alice with age 30]
Batch Job execution completed.
```
Run the AddPeopleWriterTest.java file.
1. Find the AddDescPeopleWriterTest.java file in src > test > java in the project structure.
2. In the tool menu, choose Run > Run... > AddPeopleWriterTest.testWrite, or click the green triangle in the upper-right corner to run it.
3. View the log information and output results in the console of IntelliJ IDEA.
```
Data in the people table:
People [name=zhangsan, age=27]
People [name=lisi, age=35]
Batch Job execution completed.
```

Project code

Click java-oceanbase-springbatch to download the project code, which is a compressed file named java-oceanbase-springbatch.

After decompressing the file, you will find a folder named java-oceanbase-springbatch. The directory structure is as follows:

│  pom.xml
│
├─.idea
│
├─src
│  ├─main
│  │  ├─java
│  │  │  └─com
│  │  │      └─oceanbase
│  │  │          └─example
│  │  │              └─batch
│  │  │                  │──BatchApplication.java
│  │  │                  │
│  │  │                  ├─config
│  │  │                  │   └─BatchConfig.java
│  │  │                  │
│  │  │                  ├─model
│  │  │                  │   ├─People.java
│  │  │                  │   └─PeopleDESC.java
│  │  │                  │
│  │  │                  ├─processor
│  │  │                  │   └─AddPeopleDescProcessor.java
│  │  │                  │
│  │  │                  └─writer
│  │  │                      ├─AddDescPeopleWriter.java
│  │  │                      └─AddPeopleWriter.java
│  │  │
│  │  └─resources
│  │      └─application.properties
│  │
│  └─test
│      └─java
│          └─com
│              └─oceanbase
│                  └─example
│                      └─batch
│                          ├─config
│                          │   └─BatchConfigTest.java
│                          │
│                          ├─processor
│                          │   └─AddPeopleDescProcessorTest.java
│                          │
│                          └─writer
│                              ├─AddDescPeopleWriterTest.java
│                              └─AddPeopleWriterTest.java
│
└─target

File description:

pom.xml: the configuration file of the Maven project, which contains information about the project's dependencies, plugins, and build process.
.idea: the directory used by the IDE (Integrated Development Environment) to store project-related configuration information.
src: the directory typically used to store the source code of the project.
main: the directory for storing the main source code and resource files.
java: the directory for storing the Java source code.
com.oceanbase.example.batch: the package name.
BatchApplication.java: the entry class of the application, which contains the main method of the application.
config: the folder for storing the configuration class files of the application.
BatchConfig.java: the configuration class of the application, used to configure some properties and behaviors of the application.
model: the folder for storing the model class files of the application.
People.java: the data model class for people.
PeopleDESC.java: the data model class for people's description.
processor: the folder for storing the processor class files of the application.
AddPeopleDescProcessor.java: the processor class for adding people's description information.
writer: the folder for storing the writer class files of the application.
AddDescPeopleWriter.java: the writer class for writing people's description information.
AddPeopleWriter.java: the writer class for writing people's information.
resources: the folder for storing the resource files of the application, including the configuration files and other static resources.
application.properties: the configuration file of the application, used to configure the properties of the application.
test: the directory for storing the test code and resource files.
BatchConfigTest.java: the test class of the application configuration class.
AddPeopleDescProcessorTest.java: the test class of the processor for adding people's description information.
AddDescPeopleWriterTest.java: the test class of the writer for writing people's description information.
AddPeopleWriterTest.java: the test class of the writer for writing people's information.
target: the directory for storing the compiled Class files, Jar packages, and other files.

Introduction to the pom.xml file

Note

If you only want to verify the sample, use the default code without any modifications. You can also modify the pom.xml file based on your requirements as described below.

The content of the pom.xml configuration file is as follows:

File declaration statement.

Declares that this file is an XML file using XML version 1.0 and character encoding UTF-8.

Sample code:
```
<?xml version="1.0" encoding="UTF-8"?>
```
Configure the namespaces and POM model version.
1. Use xmlns to set the POM namespace to http://maven.apache.org/POM/4.0.0.
2. Use xmlns:xsi to set the XML namespace to http://www.w3.org/2001/XMLSchema-instance.
3. Use xsi:schemaLocation to set the POM namespace to http://maven.apache.org/POM/4.0.0 and the location of the POM XSD file to https://maven.apache.org/xsd/maven-4.0.0.xsd.
4. Use the <modelVersion> element to set the POM model version used by this POM file to 4.0.0.
Sample code:
```
 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
     <modelVersion>4.0.0</modelVersion>
</project>
```
Configure the parent project information.
1. Use <groupId> to set the parent project identifier to org.springframework.boot.
2. Use <artifactId> to set the parent project dependency to spring-boot-starter-parent.
3. Use <version> to set the parent project version to 2.7.11.
4. Use relativePath to set the parent project path to empty.
Sample code:
```
 <parent>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-parent</artifactId>
     <version>2.7.11</version>
     <relativePath/>
 </parent>
```
Configure basic information.
1. Use <groupId> to set the project identifier to com.oceanbase.
2. Use <artifactId> to set the project dependency to java-oceanbase-springboot.
3. Use <version> to set the project version to 0.0.1-SNAPSHOT.
4. Use description to set the project information to Demo project for Spring Batch.
Sample code:
```
 <groupId>com.oceanbase</groupId>
 <artifactId>java-oceanbase-springboot</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <name>java-oceanbase-springbatch</name>
 <description>Demo project for Spring Batch</description>
```
Configure the Java version.

Set the Java version used by the project to 1.8.

Sample code:
```
  <properties>
      <java.version>1.8</java.version>
  </properties>
```
Configure core dependencies.
1. Set the organization of the dependency to org.springframework.boot, the name to spring-boot-starter, and use this dependency to access the component dependencies supported by Spring Boot by default. This dependency supports features such as Web, data processing, security, and Test.
  
  Sample code:
```
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
</dependency>
```
2. Set the organization of the dependency to org.springframework.boot, the name to spring-boot-starter-jdbc, and use this dependency to access the JDBC-related features provided by Spring Boot, such as connection pools and data source configurations.
  
  Sample code:
```
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
```
3. Set the organization of the dependency to org.springframework.boot, the name to spring-boot-starter-test, and the scope to test. Use this dependency to access the test framework and tools provided by Spring Boot, such as JUnit, Mockito, and Hamcrest.
  
  Sample code:
```
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>
```
4. Set the organization of the dependency to com.oceanbase, the name to oceanbase-client, and the version to 2.4.12. Use this dependency to access the client features provided by OceanBase, such as connections, queries, and transactions.
  
  Sample code:
```
    <dependency>
        <groupId>com.oceanbase</groupId>
        <artifactId>oceanbase-client</artifactId>
        <version>2.4.12</version>
    </dependency>
```
5. Set the organization of the dependency to org.springframework.boot, the name to spring-boot-starter-batch, and use this dependency to access the batch processing features provided by Spring Boot.
  
  Sample code:
```
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
</dependency>
```
6. Set the organization of the dependency to org.springframework.boot, the name to spring-boot-starter-data-jpa, and use this dependency to access the necessary dependencies and configurations for data access using JPA. Spring Boot Starter Data JPA is a starter provided by Spring Boot.
  
  Sample code:
```
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
```
7. Set the organization of the dependency to org.apache.tomcat, the name to tomcat-jdbc, and use this dependency to access the JDBC connection pool features provided by Tomcat, including connection pool configuration, connection acquisition and release, and connection management.
  
  Sample code:
```
<dependency>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>tomcat-jdbc</artifactId>
</dependency>
```
8. Set the test framework of the dependency to junit, the name to junit, the version to 4.10, and the scope to test. Use this dependency to add the configuration for the JUnit unit test dependency.
  
  Sample code:
```
<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.10</version>
    <scope>test</scope>
</dependency>
```
9. Set the organization of the dependency to javax.activation, the name to javax.activation-api, and the version to 1.2.0. Use this dependency to introduce the Java Activation Framework (JAF) library.
  
  Sample code:
```
<dependency>
    <groupId>javax.activation</groupId>
    <artifactId>javax.activation-api</artifactId>
    <version>1.2.0</version>
</dependency>
```
10. Set the organization of the dependency to jakarta.persistence, the name to jakarta.persistence-api, and the version to 2.2.3. Use this dependency to add the configuration for the Jakarta Persistence API dependency. Sample code:
```
<dependency>
    <groupId>jakarta.persistence</groupId>
    <artifactId>jakarta.persistence-api</artifactId>
    <version>2.2.3</version>
</dependency>
```

Configure Maven plugins.

Set the organization of the dependency to org.springframework.boot, the name to spring-boot-maven-plugin, and use this plugin to package the Spring Boot application into an executable JAR or WAR file, which can be directly run.

Sample code:

 <build>
     <plugins>
         <plugin>
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-maven-plugin</artifactId>
         </plugin>
     </plugins>
 </build>

Introduction to the application.properties file

The application.properties file is used to configure database connections and other related settings, such as the database driver, connection URL, username, and password. It also includes configurations for JPA (Java Persistence API), Spring Batch, and log levels.

Database connection configuration.
- The spring.datasource.driver parameter specifies the database driver as com.mysql.cj.jdbc.Driver, which is used to establish a connection to OceanBase Database.
- The spring.datasource.url parameter specifies the URL for connecting to the database.
- The spring.datasource.username parameter specifies the username for connecting to the database.
- The spring.datasource.password parameter specifies the password for connecting to the database.
Sample code:
```
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://host:port/schema_name?characterEncoding=utf-8
spring.datasource.username=user_name
spring.datasource.password=******
```
JPA configuration.
- The spring.jpa.show-sql parameter specifies whether to display SQL statements in the logs. Setting it to true means SQL statements will be displayed.
- The spring.jpa.hibernate.ddl-auto parameter specifies the Hibernate DDL operation behavior. Here, it is set to update, which means the database structure will be automatically updated when the application starts.
Sample code:
```
spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update
```
Spring Batch configuration:

The spring.batch.job.enabled parameter specifies whether to enable Spring Batch jobs. Here, it is set to false, indicating that automatic execution of batch jobs is disabled.

Sample code:
```
spring.batch.job.enabled=false
```
Note

In Spring Batch, the spring.batch.job.enabled property controls the execution behavior of batch jobs.
- spring.batch.job.enabled=true (default): Indicates that all defined batch jobs will be automatically executed when the Spring Boot application starts. This means that Spring Batch will automatically discover and execute all defined jobs when the application starts.
- spring.batch.job.enabled=false: Indicates that automatic execution of batch jobs is disabled. This is typically used in development or testing environments, or when you want to manually control job execution. When set to false, jobs will not be automatically executed when the application starts. You can manually trigger jobs using other methods, such as REST APIs or command-line tools.
In summary, setting spring.batch.job.enabled=false allows you to avoid automatically running jobs when the application starts, providing greater flexibility in controlling when batch jobs are executed.
Log configuration:
- The logging.level.org.springframework parameter specifies the log level for the Spring framework as INFO.
- The logging.level.com.example parameter specifies the log level for custom application code as DEBUG.
Sample code:
```
logging.level.org.springframework=INFO
logging.level.com.example=DEBUG
```

Introduction to the BatchApplication.java file

The BatchApplication.java file is the entry point of the Spring Boot application.

The code in the BatchApplication.java file mainly includes the following parts:

Importing other classes and interfaces.

The current file includes the following interfaces and classes:
- SpringApplication class: used to start the Spring Boot application.
- SpringBootApplication annotation: used to mark the class as the entry point of the Spring Boot application.
Sample code:
```
    import org.springframework.boot.SpringApplication;
    import org.springframework.boot.autoconfigure.SpringBootApplication;
```
Defining the BatchApplication class.

The @SpringBootApplication annotation is used to mark the BatchApplication class as the entry point of the Spring Boot application. The BatchApplication class defines a static main method as the entry point of the application. In this method, the SpringApplication.run method is used to start the Spring Boot application. A method named runBatchJob is also defined to run batch jobs.

Sample code:
```
    @SpringBootApplication
    public class BatchApplication {
        public static void main(String[] args) {
            SpringApplication.run(BatchApplication.class, args);
        }

        public void runBatchJob() {
        }
    }
```

Introduction to the BatchConfig.java file

The BatchConfig.java file is used to configure components such as steps, readers, processors, and writers for batch processing jobs.

The code in the BatchConfig.java file mainly includes the following parts:

Import other classes and interfaces.

Declare the interfaces and classes included in the current file:
- People class: used to store personnel information read from the database.
- PeopleDESC class: used to store description information after the personnel information is converted or processed.
- AddPeopleDescProcessor class: converts the read People object to a PeopleDESC object. It is an implementation class of the ItemProcessor interface.
- AddDescPeopleWriter class: writes the PeopleDESC object to the target location. It is an implementation class of the ItemWriter interface.
- Job interface: represents a batch processing job.
- Step interface: represents a step in a job.
- EnableBatchProcessing annotation: a Spring Batch configuration annotation used to enable and configure Spring Batch processing.
- JobBuilderFactory class: used to create and configure jobs.
- StepBuilderFactory class: used to create and configure steps.
- RunIdIncrementer class: a Spring Batch run ID (Run ID) incrementer used to increment the run ID each time a job is run.
- ItemProcessor interface: used to process or convert read items.
- ItemReader interface: used to read items from a data source.
- ItemWriter interface: used to write processed or converted items to the specified target location.
- JdbcCursorItemReader class: used to read data from the database and return a cursor result set.
- Autowired annotation: used for dependency injection.
- Bean annotation: used to create and configure beans.
- ComponentScan annotation: used to specify the package or class to be scanned for components.
- Configuration annotation: used to mark a class as a configuration class.
- EnableAutoConfiguration annotation: used to enable Spring Boot auto-configuration.
- SpringBootApplication annotation: used to mark the class as the entry point of a Spring Boot application.
- DataSource interface: used to represent a database connection.
Sample code:
```
import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import com.oceanbase.example.batch.processor.AddPeopleDescProcessor;
import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.database.JdbcCursorItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.BeanPropertyRowMapper;

import javax.sql.DataSource;
```
Define the BatchConfig class.

This is a simple Spring Batch batch processing job. It defines the methods for reading, processing, and writing data, and encapsulates these steps into a single job. Using Spring Batch annotations and auto-configuration features, you can create corresponding component instances through the @Bean methods in the configuration class and use these components to complete data reading, processing, and writing in the step1 method.
- Use the @Configuration annotation to indicate that this class is a configuration class.
- Use the @EnableBatchProcessing annotation to enable Spring Batch processing. This annotation automatically creates necessary beans, such as JobRepository and JobLauncher.
- Use the @SpringBootApplication annotation for the main class of a Spring Boot application, which serves as the entry point for the Spring Boot application.
- Use the @ComponentScan annotation to specify the package to be scanned for components, telling Spring to scan and register all components in this package and its subpackages.
- Use the @EnableAutoConfiguration annotation to automatically configure the infrastructure of the Spring Boot application.
Sample code:
```
 @Configuration
 @EnableBatchProcessing
 @SpringBootApplication
 @ComponentScan("com.oceanbase.example.batch.writer")
 @EnableAutoConfiguration
 public class BatchConfig {
 }
```
1. Define the @Autowired annotation.
  
  Use the @Autowired annotation to inject the JobBuilderFactory, StepBuilderFactory, and DataSource into the member variables of the BatchConfig class. The JobBuilderFactory is a factory class used to create and configure jobs (Job), the StepBuilderFactory is a factory class used to create and configure steps (Step), and the DataSource is an interface used to obtain a database connection.
  
  Sample code:
```
@Autowired
private JobBuilderFactory jobBuilderFactory;

@Autowired
private StepBuilderFactory stepBuilderFactory;

@Autowired
private DataSource dataSource;
```
2. Define the @Bean annotation.
  
  Use the @Bean annotation to define several methods for creating the reader, processor, writer, step, and job components of a batch processing job.
  - Use the peopleReader method to create an ItemReader component instance. This component uses JdbcCursorItemReader to read People object data from the database. Set the data source to dataSource, set the RowMapper to map database rows to People objects, and set the SQL query statement to SELECT * FROM people.
  - Use the addPeopleDescProcessor method to create an ItemProcessor component instance. This component uses AddPeopleDescProcessor to process People objects and returns the converted PeopleDESC objects.
  - Use the addDescPeopleWriter method to create an ItemWriter component instance. This component uses AddDescPeopleWriter to write PeopleDESC objects to the target location.
  - Use the step1 method to create a Step component instance. The step name is step1. Use stepBuilderFactory.get to obtain the step builder, set the reader to the ItemReader component, set the processor to the ItemProcessor component, set the writer to the ItemWriter component, set the chunk size to 10, and finally call build to build and return the configured Step.
  - Use the importJob method to create a Job component instance. The job name is importJob. Use jobBuilderFactory.get to obtain the job builder, set the incrementer to RunIdIncrementer, set the initial step of the job flow to Step, and finally call build to build and return the configured Job.
    
    Sample code:
```
@Bean
public ItemReader<People> peopleReader() {
    JdbcCursorItemReader<People> reader = new JdbcCursorItemReader<>();
    reader.setDataSource((javax.sql.DataSource) dataSource);
    reader.setRowMapper(new BeanPropertyRowMapper<>(People.class));
    reader.setSql("SELECT * FROM people");
    return reader;
}

@Bean
public ItemProcessor<People, PeopleDESC> addPeopleDescProcessor() {
    return new AddPeopleDescProcessor();
}

@Bean
public ItemWriter<PeopleDESC> addDescPeopleWriter() {
    return new AddDescPeopleWriter();
}

@Bean
public Step step1(ItemReader<People> reader, ItemProcessor<People, PeopleDESC> processor,
                ItemWriter<PeopleDESC> writer) {
    return stepBuilderFactory.get("step1")
            .<People, PeopleDESC>chunk(10)
            .reader(reader)
            .processor(processor)
            .writer(writer)
            .build();
}

@Bean
public Job importJob(Step step1) {
    return jobBuilderFactory.get("importJob")
            .incrementer(new RunIdIncrementer())
            .flow(step1)
            .end()
            .build();
}
```

Introduction to the People.java file

The People.java file defines a data model for the People class, which represents information about a person. This class contains two private member variables, name and age, along with corresponding getter and setter methods. The toString method is overridden to print the information of the object. Here, name represents the person's name, and age represents the person's age. You can use the getter and setter methods to obtain and set the values of these attributes.

The People class is used to store and pass data in the input and output of the batch processing program. In the batch processing program, the People class is used to store data. You can use the setter method to set data and the getter method to obtain data.

Code:

    public class People {
        private String name;
        private int age;

            // getters and setters

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public int getAge() {
            return age;
        }

        public void setAge(int age) {
            this.age = age;
        }
        @Override
        public String toString() {
            return "People [name=" + name + ", age=" + age + "]";
        }
        // Getters and setters
    }

Introduction to the PeopleDESC.java file

The PeopleDESC.java file defines a data model for the PeopleDESC class, which represents information about a person. The PeopleDESC class has four attributes: name, age, desc, and id, which represent the person's name, age, description, and identifier, respectively. This class contains corresponding getter and setter methods to access and set the values of the attributes. The toString method is overridden to return the string representation of the class, which includes the name, age, and description.

Similar to the People class, the PeopleDESC class is used to store and pass data in the input and output of the batch processing program.

Code:

    public class PeopleDESC {
        private String name;
        private int age;
        private String desc;
        private int id;

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public int getAge() {
            return age;
        }

        public void setAge(int age) {
            this.age = age;
        }

        public String getDesc() {
            return desc;
        }

        public void setDesc(String desc) {
            this.desc = desc;
        }

        public int getId() {
            return id;
        }

        public void setId(int id) {
            this.id = id;
        }

        @Override
        public String toString() {
            return "PeopleDESC [name=" + name + ", age=" + age + ", desc=" + desc + "]";
        }
    }

Introduction to the AddPeopleDescProcessor.java file

The AddPeopleDescProcessor.java file defines a class named AddPeopleDescProcessor that implements the ItemProcessor interface. This class is used to convert a People object to a PeopleDESC object.

The code in the AddPeopleDescProcessor.java file mainly includes the following parts:

Import other classes and interfaces.

The following interfaces and classes are included in this file:
- People class: used to store the information of a person read from the database.
- PeopleDESC class: used to store the description information of a person after the information is converted or processed.
- ItemProcessor interface: used to process or convert the read item.
Code:
```
import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemProcessor;
```
Define the AddPeopleDescProcessor class.

The AddPeopleDescProcessor class of the ItemProcessor interface is used to convert a People object to a PeopleDESC object. This class implements the logic for processing the input data in the batch processing program.

In the process method of this class, a PeopleDESC object desc is first created. Then, the item parameter is used to obtain the name and age attributes of the People object and set these attributes to the desc object. The desc attribute of the desc object is also set. The logic for setting the desc attribute is to generate a description based on the attributes of the People object. Finally, the processed PeopleDESC object is returned.

Code:
```
public class AddPeopleDescProcessor implements ItemProcessor<People, PeopleDESC> {
    @Override
    public PeopleDESC process(People item) throws Exception {
        PeopleDESC desc = new PeopleDESC();
        desc.setName(item.getName());
        desc.setAge(item.getAge());
        desc.setDesc("This is " + item.getName() + " with age " + item.getAge());
        return desc;
    }
}
```

Introduction to the AddDescPeopleWriter.java file

The AddDescPeopleWriter.java file implements the AddDescPeopleWriter class that implements the ItemWriter interface. This class is used to write People objects to a database.

The code in the AddDescPeopleWriter.java file mainly includes the following parts:

Import other classes and interfaces.

Declare the following interfaces and classes in the current file:
- PeopleDESC class: used to store the description information of the personnel after conversion or processing.
- ItemWriter interface: used to write the processed or converted items to the specified target location.
- @Autowired annotation: used for dependency injection.
- JdbcTemplate class: provides methods for executing SQL statements.
- List interface: used to operate on the query result set.
Code:
```
import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;

import java.util.List;
```

Define the AddDescPeopleWriter class.

Use the @Autowired annotation to automatically inject the JdbcTemplate instance. This instance is used to execute database operations when writing data.

Code:
```
    @Autowired
    private JdbcTemplate jdbcTemplate;
```

In the write method, traverse the input List<? extends PeopleDESC> and extract each PeopleDESC object. First, execute the SQL statement DROP TABLE people_desc to delete the table named people_desc if it exists. Then, execute the SQL statement CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255)) to create a table named people_desc with four columns: id, name, age, and description. Finally, use the SQL statement INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?) to insert the attribute values of each PeopleDESC object into the people_desc table.

Code:

    @Override
    public void write(List<? extends PeopleDESC> items) throws Exception {
        // Drop the table if it exists
        jdbcTemplate.execute("DROP TABLE people_desc");
        // Create the table
        String createTableSql = "CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255))";
        jdbcTemplate.execute(createTableSql);
        for (PeopleDESC item : items) {
            String sql = "INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?)";
            jdbcTemplate.update(sql, item.getId(), item.getName(), item.getAge(), item.getDesc());
        }
    }

AddPeopleWriter.java file

The AddPeopleWriter.java file implements the AddDescPeopleWriter class of the ItemWriter interface. It is used to write PeopleDESC objects to a database.

The AddPeopleWriter.java file contains the following code:

Import other classes and interfaces.

The following interfaces and classes are declared in this file:
- People class: used to store personnel information read from a database.
- ItemWriter interface: used to write processed or converted items to a specified target location.
- @Autowired annotation: used for dependency injection.
- JdbcTemplate class: provides methods for executing SQL statements.
- @Component annotation: used to mark the class as a Spring component.
- List interface: used to operate on query result sets.
Sample code:
```
import com.oceanbase.example.batch.model.People;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;

import java.util.List;
```
Define the AddPeopleWriter class.
1. Use the @Autowired annotation to automatically inject the JdbcTemplate instance. This instance is used to execute database operations when data is written.
  
  Sample code:
```
    @Autowired
    private JdbcTemplate jdbcTemplate;
```
2. In the write method, traverse the input List<? extends People> and extract each People object. First, execute the SQL statement DROP TABLE people to delete the people table if it exists. Then, execute the SQL statement CREATE TABLE people (name VARCHAR2(255), age INT) to create a people table with name and age columns. Finally, use the SQL statement INSERT INTO people (name, age) VALUES (?, ?) to insert the attribute values of each People object into the people table.
  
  Sample code:
```
@Override
public void write(List<? extends People> items) throws Exception {
    // Drop the table if it exists
    jdbcTemplate.execute("DROP TABLE people");
    // Create the table
    String createTableSql = "CREATE TABLE people (name VARCHAR2(255), age INT)";
    jdbcTemplate.execute(createTableSql);
    for (People item : items) {
        String sql = "INSERT INTO people (name, age) VALUES (?, ?)";
        jdbcTemplate.update(sql, item.getName(), item.getAge());
    }
}
```

BatchConfigTest.java file

The BatchConfigTest.java file is a class that uses JUnit for testing. It is used to test the job configuration of Spring Batch.

The BatchConfigTest.java file contains the following code:

Import other classes and interfaces.

The following interfaces and classes are declared in this file:
- Assert class: used to assert test results.
- @Test annotation: used to mark a method as a test method.
- @RunWith annotation: used to specify the test runner.
- Job interface: represents a batch processing job.
- JobExecution class: used to represent the execution of a batch processing job.
- JobParameters class: used to represent the parameters of a batch processing job.
- JobParametersBuilder class: used to build the parameters of a batch processing job.
- JobLauncher interface: used to start a batch processing job.
- @Autowired annotation: used for dependency injection.
- @SpringBootTest annotation: used to specify the test class as a Spring Boot test.
- SpringRunner class: used to specify the test runner as SpringRunner.
Sample code:
```
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import javax.batch.runtime.BatchStatus;
import java.util.UUID;
```
Define the BatchConfigTest class.

By using the @SpringBootTest annotation and the SpringRunner runner, you can perform integration tests for Spring Boot. In the testJob method, use the JobLauncherTestUtils helper class to start the batch processing job and use assertions to verify the job's execution status.
1. Use the @Autowired annotation to automatically inject the JobLauncherTestUtils instance.
  
  Sample code:
```
@Autowired
private JobLauncherTestUtils jobLauncherTestUtils;
```
2. Use the @Test annotation to mark the testJob method as a test method. In this method, first create a JobParameters object, then use the jobLauncherTestUtils.launchJob method to start the batch processing job, and use the Assert.assertEquals method to assert that the job's execution status is COMPLETED.
  
  Sample code:
```
@Test
public void testJob() throws Exception {
    JobParameters jobParameters = new JobParametersBuilder()
            .addString("jobParam", "paramValue")
            .toJobParameters();

    JobExecution jobExecution = jobLauncherTestUtils.launchJob(jobParameters);

    Assert.assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus());
}
```
3. Use the @Autowired annotation to automatically inject the JobLauncher instance.
  
  Sample code:
```
@Autowired
private JobLauncher jobLauncher;
```
4. Use the @Autowired annotation to automatically inject the Job instance.
  
  Sample code:
```
@Autowired
private Job job;
```
5. Define an internal class named JobLauncherTestUtils to assist in starting the batch processing job. In this class, define a launchJob method to start the batch processing job. In this method, use the jobLauncher.run method to start the job and return the job's execution result.
  
  Sample code:
```
private class JobLauncherTestUtils {
    public JobExecution launchJob(JobParameters jobParameters) throws Exception {
        return jobLauncher.run(job, jobParameters);
    }
}
```

AddPeopleDescProcessorTest.java file

The AddPeopleDescProcessorTest.java file is a class that uses JUnit for testing Spring Batch job configurations.

The AddPeopleDescProcessorTest.java file contains the following code:

Import other classes and interfaces.

The following interfaces and classes are declared in this file:
- People class: used to store the information of a person read from the database.
- PeopleDESC class: used to store the description information of a person after conversion or processing.
- Assert class: used to verify whether the expected results and actual results in the test are consistent.
- Test annotation: used to mark a method as a test method.
- RunWith annotation: used to specify the test runner.
- Autowired annotation: used for dependency injection.
- SpringBootTest annotation: used to specify the test class as a Spring Boot test class.
- SpringRunner class: used to specify the test runner as SpringRunner.
Sample code:
```
import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;
```
Define the AddPeopleDescProcessorTest class.

Use the SpringBootTest annotation and SpringRunner runner to perform Spring Boot integration testing.
1. Use the @Autowired annotation to automatically inject the AddPeopleDescProcessor instance.
  
  Sample code:
```
@Autowired
private AddPeopleDescProcessor processor;
```
2. Use the @Test annotation to mark the testProcess method as a test method. In this method, first create a People object, then use the processor.process method to process this object, and assign the result to a PeopleDESC object.
  
  Sample code:
```
@Test
public void testProcess() throws Exception {
People people = new People();
people.setName("John");
people.setAge(25);

PeopleDESC desc = processor.process(people);
}
```

File description: AddDescPeopleWriterTest.java

The AddDescPeopleWriterTest.java file is a class that uses JUnit to test the write logic of the AddDescPeopleWriter class.

The AddDescPeopleWriterTest.java file contains the following code segments:

Import another class or interface.

Declares that the current file contains the following interfaces and classes:
- PeopleDESC class: stores description information after personnel information has been converted or processed.
- Assert class: Used to assert test results.
- Test annotation: This annotation is used to mark test methods.
- RunWith: specifies the runner to be used for running the test.
- @Autowired annotation: used for dependency injection.
- @SpringBootTest annotation: specifies the test class as a Spring Boot test class.
- JdbcTemplate class: Provides methods to execute SQL statements.
- SpringRunner class: specifies SpringRunner as the test runner.
- ArrayList Class to create an empty list.
- List interface: Operations that you can perform on query result sets.
Sample code:
```
import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;
```
Define the AddDescPeopleWriterTest class.

You can perform integration tests for Spring Boot by using the SpringBootTest annotation and the SpringRunner runner.
1. Inject the instance with the @Autowired annotation. Autowire the AddPeopleDescProcessor and JdbcTemplate instances by using the @Autowired annotation.
  
  Sample code:
```
@Autowired
private AddDescPeopleWriter writer;
@Autowired
private JdbcTemplate jdbcTemplate;
```
2. Run test cases by using the @Test method. The testWrite method is marked as a test method with the @Test annotation. In this method, an empty peopleDescList list is first created and two PeopleDESC objects are added to the list. Then, the writer.write method is used to write the list data to the database. Subsequently, the jdbcTemplate is used to execute a query statement to retrieve data from the people_desc table, and assertion statements are used to verify the correctness of the data. Finally, the query results are output to the console, along with a message indicating that the job has been completed.
  1. Insert data into the people_desc table. First, an empty PeopleDESC object list named peopleDescList is created. Then, two PeopleDESC objects named desc1 and desc2 are created, and their attribute values are set. desc1 and desc2 are added to peopleDescList. Then, the write method of writer is called to write the objects in peopleDescList to the people_desc table in the database. Next, a JdbcTemplate is used to execute a query statement SELECT COUNT(*) FROM people_desc to get the number of records in the people_desc table and assign the result to the count variable. Finally, the Assert.assertEquals method is used to verify that the value of count is 2.
    
    The code is as follows:
```
   List<PeopleDESC> peopleDescList = new ArrayList<>();
   PeopleDESC desc1 = new PeopleDESC();
   desc1.setId(1);
   desc1.setName("John");
   desc1.setAge(25);
   desc1.setDesc("This is John with age 25");
   peopleDescList.add(desc1);
   PeopleDESC desc2 = new PeopleDESC();
   desc2.setId(2);
   desc2.setName("Alice");
   desc2.setAge(30);
   desc2.setDesc("This is Alice with age 30");
   peopleDescList.add(desc2);
   writer.write(peopleDescList);

   String selectSql = "SELECT COUNT(*) FROM people_desc";
   int count = jdbcTemplate.queryForObject(selectSql, Integer.class);
   Assert.assertEquals(2, count);
```
  2. Output the data in the people_desc table. First, a JdbcTemplate is used to execute the query statement SELECT * FROM people_desc, and lambda expressions are used to process the query results. In the lambda expression, methods such as rs.getInt and rs.getString are used to retrieve field values from the query result set, and these values are set in a newly created PeopleDESC object. Each new PeopleDESC object is added to a result list resultDesc. Then, a line of prompt information people_desc table data: is printed. The for loop is used to iterate through each PeopleDESC object in the resultDesc list, and System.out.println is used to print the contents of each object. Finally, a message indicating the completion of the job is printed.
    
    Code example:
```
List<PeopleDESC> resultDesc = jdbcTemplate.query("SELECT * FROM people_desc", (rs, rowNum) -> {
   PeopleDESC desc = new PeopleDESC();
   desc.setId(rs.getInt("id"));
   desc.setName(rs.getString("name"));
   desc.setAge(rs.getInt("age"));
   desc.setDesc(rs.getString("description"));
   return desc;
});

System.out.println("Data in the people_desc table:");
for (PeopleDESC desc : resultDesc) {
   System.out.println(desc);
}

// Output information after the job finishes.
System.out.println("Batch Job execution completed.");
```

AddPeopleWriterTest.java File Description

The AddPeopleWriterTest.java file is a class that uses JUnit for testing the write logic of AddPeopleWriterTest.

The AddPeopleWriterTest.java file mainly contains the following code:

Import other classes and interfaces.

This file contains the following interfaces and classes:
- People class: Stores the information of people retrieved from the database.
- Test annotation: Marks a method as a test method.
- RunWith annotation: specifies the test runner.
- Autowired annotation: used for dependency injection.
- SpringBootApplication annotation: This annotation identifies the class as an entry point of a Spring Boot application.
- SpringBootTest annotation: specifies a test class as a Spring Boot test class.
- ComponentScan annotation: specifies the package or class for component scan.
- JdbcTemplate class: provides methods to execute SQL statements.
- SpringRunner class: specifies SpringRunner as the test runner.
- ArrayList class, which is used to create an empty list.
- The List interface is used for handling the result set of a query.
Sample code:
```
import com.oceanbase.example.batch.model.People;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;
```
Define the AddPeopleWriterTest class.

Spring Boot integration tests are performed using SpringBootTest annotation and SpringRunner runner and @ComponentScan annotation is used to specify the package path to be scanned.
1. Use @Autowired to inject an instance. Use the @Autowired annotation to automatically inject instances of addPeopleWriter and JdbcTemplate.
  
  Sample code:
```
@Autowired
private AddPeopleWriter addPeopleWriter;
@Autowired
private JdbcTemplate jdbcTemplate;
```
2. Insert and query test data by using the @Test method.
  1. Insert data to the people table. First, create an empty list of People objects called peopleList. Then, create two People objects named person1 and person2, and set their name and age properties. Next, add these two People objects to the peopleList list. Finally, call the write method of addPeopleWriter and pass the peopleList list as the parameter to the method to write the People objects to the database.
    
    Here is the code:
```
   List<People> peopleList = new ArrayList<>();
   People person1 = new People();
   person1.setName("zhangsan");
   person1.setAge(27);
   peopleList.add(person1);
   People person2 = new People();
   person2.setName("lisi");
   person2.setAge(35);
   peopleList.add(person2);
   addPeopleWriter.write(peopleList);
```
  2. Output the data in the people table. JdbcTemplate is first used to execute a query statement SELECT * FROM people. The result is processed by using a lambda expression. In the lambda expression, the rs.getString and rs.getInt methods are used to obtain field values in the query result set. The field values are set to a new People object. Each new People object is added to a result list result. Then, a prompt message people table data: is printed. The for loop is used to traverse each People object in the result list, and the System.out.println method is used to output the contents of each object. Finally, a message indicating that the task is completed is printed.
    
    Code example:
```
   List<People> result = jdbcTemplate.query("SELECT * FROM people", (rs, rowNum) -> {
       People person = new People();
       person.setName(rs.getString("name"));
       person.setAge(rs.getInt("age"));
       return person;
   });

   System.out.println("people table:");
   for (People person : result) {
       System.out.println(person);
   }

   // Outputs information after the job is executed.
   System.out.println("Batch Job execution completed.");
```

Complete code

pom.xml

application.properties

BatchApplication.java

BatchConfig.java

People.java

PeopleDESC.java

AddPeopleDescProcessor.java

AddDescPeopleWriter.java

AddPeopleWriter.java

BatchConfigTest.java

AddPeopleDescProcessorTest.java

AddDescPeopleWriterTest.java

AddPeopleWriterTest.java

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.11</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.oceanbase</groupId>
    <artifactId>java-oceanbase-springboot</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>java-oceanbase-springbatch</name>
    <description>Demo project for Spring Batch</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>com.oceanbase</groupId>
            <artifactId>oceanbase-client</artifactId>
            <version>2.4.3</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-batch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.tomcat</groupId>
            <artifactId>tomcat-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>javax.activation</groupId>
            <artifactId>javax.activation-api</artifactId>
            <version>1.2.0</version>
        </dependency>
        <dependency>
            <groupId>jakarta.persistence</groupId>
            <artifactId>jakarta.persistence-api</artifactId>
            <version>2.2.3</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

#configuration database

spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://host:port/schema_name?characterEncoding=utf-8
spring.datasource.username=user_name
spring.datasource.password=

# JPA
spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update

# Spring Batch
spring.batch.job.enabled=false

#
logging.level.org.springframework=INFO
logging.level.com.example=DEBUG

package com.oceanbase.example.batch;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class BatchApplication {
    public static void main(String[] args) {
        SpringApplication.run(BatchApplication.class, args);
    }

    public void runBatchJob() {
    }
}

package com.oceanbase.example.batch.config;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import com.oceanbase.example.batch.processor.AddPeopleDescProcessor;
import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.database.JdbcCursorItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.BeanPropertyRowMapper;

import javax.sql.DataSource;
//import javax.activation.DataSource;

@Configuration
@EnableBatchProcessing
@SpringBootApplication
@ComponentScan("com.oceanbase.example.batch.writer")
@EnableAutoConfiguration
public class BatchConfig {
    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Autowired
    private DataSource dataSource;// Use the default dataSource provided by Spring Boot auto-configuration.

    @Bean
    public ItemReader<People> peopleReader() {
        JdbcCursorItemReader<People> reader = new JdbcCursorItemReader<>();
        reader.setDataSource((javax.sql.DataSource) dataSource);
        reader.setRowMapper(new BeanPropertyRowMapper<>(People.class));
        reader.setSql("SELECT * FROM people");
        return reader;
    }

    @Bean
    public ItemProcessor<People, PeopleDESC> addPeopleDescProcessor() {
        return new AddPeopleDescProcessor();
    }

    @Bean
    public ItemWriter<PeopleDESC> addDescPeopleWriter() {
        return new AddDescPeopleWriter();
    }

    @Bean
    public Step step1(ItemReader<People> reader, ItemProcessor<People, PeopleDESC> processor,
                      ItemWriter<PeopleDESC> writer) {
        return stepBuilderFactory.get("step1")
                .<People, PeopleDESC>chunk(10)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .build();
    }

    @Bean
    public Job importJob(Step step1) {
        return jobBuilderFactory.get("importJob")
                .incrementer(new RunIdIncrementer())
                .flow(step1)
                .end()
                .build();
    }
}

package com.oceanbase.example.batch.model;

public class People {
    private String name;
    private int age;

        // getters and setters

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }
    @Override
    public String toString() {
        return "People [name=" + name + ", age=" + age + "]";
    }
    // Getters and setters
}

package com.oceanbase.example.batch.model;

public class PeopleDESC {
    private String name;
    private int age;
    private String desc;
    private int id;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public String getDesc() {
        return desc;
    }

    public void setDesc(String desc) {
        this.desc = desc;
    }

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    @Override
    public String toString() {
        return "PeopleDESC [name=" + name + ", age=" + age + ", desc=" + desc + "]";
    }
}

package com.oceanbase.example.batch.processor;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemProcessor;

public class AddPeopleDescProcessor implements ItemProcessor<People, PeopleDESC> {
    @Override
    public PeopleDESC process(People item) throws Exception {
        PeopleDESC desc = new PeopleDESC();
        desc.setName(item.getName());
        desc.setAge(item.getAge());
        desc.setDesc("This is " + item.getName() + " with age " + item.getAge());
        return desc;
    }
}

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;

import java.util.List;

public class AddDescPeopleWriter implements ItemWriter<PeopleDESC> {
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    public void write(List<? extends PeopleDESC> items) throws Exception {
        // Drop the table if it exists.
        jdbcTemplate.execute("DROP TABLE people_desc");
        // Create the table.
        String createTableSql = "CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255))";
        jdbcTemplate.execute(createTableSql);
        for (PeopleDESC item : items) {
            String sql = "INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?)";
            jdbcTemplate.update(sql, item.getId(), item.getName(), item.getAge(), item.getDesc());
        }
    }
}

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.People;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;

import java.util.List;

@Component
public class AddPeopleWriter implements ItemWriter<People> {
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    public void write(List<? extends People> items) throws Exception {
        // Drop the table if it exists.
        jdbcTemplate.execute("DROP TABLE people");
        // Create the table.
        String createTableSql = "CREATE TABLE people (name VARCHAR2(255), age INT)";
        jdbcTemplate.execute(createTableSql);
        for (People item : items) {
            String sql = "INSERT INTO people (name, age) VALUES (?, ?)";
            jdbcTemplate.update(sql, item.getName(), item.getAge());
        }
    }
}

package com.oceanbase.example.batch.config;

import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

import org.springframework.test.context.junit4.SpringRunner;

import javax.annotation.Resource;
import javax.batch.runtime.BatchStatus;
import java.util.UUID;

@RunWith(SpringRunner.class)
@SpringBootTest
public class BatchConfigTest {

    @Test
    public void testJob() throws Exception {
        JobParameters jobParameters = new JobParametersBuilder()
                .addString("jobParam", UUID.randomUUID().toString())
                .toJobParameters();

        JobLauncherTestUtils jobLauncherTestUtils = new JobLauncherTestUtils();
        JobExecution jobExecution = jobLauncherTestUtils.launchJob(jobParameters);
        Assert.assertEquals(BatchStatus.COMPLETED.toString(), jobExecution.getStatus().toString());
    }

    @Autowired
    private JobLauncher jobLauncher;

    @Autowired
    private Job job;

    private class JobLauncherTestUtils {

        public JobExecution launchJob(JobParameters jobParameters) throws Exception {
            return jobLauncher.run(job, jobParameters);
        }
    }
}

package com.oceanbase.example.batch.processor;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
public class AddPeopleDescProcessorTest {
    @Autowired
    private AddPeopleDescProcessor processor;

    @Test
    public void testProcess() throws Exception {
        People people = new People();
  //      people.setName("John");
  //      people.setAge(25);

        PeopleDESC desc = processor.process(people);

//      Assert.assertEquals("John", desc.getName());
//        Assert.assertEquals(25, desc.getAge());
 //       Assert.assertEquals("This is John with age 25", desc.getDesc());
    }
}

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
public class AddDescPeopleWriterTest {
    @Autowired
    private AddDescPeopleWriter writer;
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Test
    public void testWrite() throws Exception {

        // Insert data into the people_desc table.
        List<PeopleDESC> peopleDescList = new ArrayList<>();
        PeopleDESC desc1 = new PeopleDESC();
        desc1.setId(1);
        desc1.setName("John");
        desc1.setAge(25);
        desc1.setDesc("This is John with age 25");
        peopleDescList.add(desc1);
        PeopleDESC desc2 = new PeopleDESC();
        desc2.setId(2);
        desc2.setName("Alice");
        desc2.setAge(30);
        desc2.setDesc("This is Alice with age 30");
        peopleDescList.add(desc2);
        writer.write(peopleDescList);

        String selectSql = "SELECT COUNT(*) FROM people_desc";
        int count = jdbcTemplate.queryForObject(selectSql, Integer.class);
        Assert.assertEquals(2, count);

        // Output the data in the people_desc table.
        List<PeopleDESC> resultDesc = jdbcTemplate.query("SELECT * FROM people_desc", (rs, rowNum) -> {
            PeopleDESC desc = new PeopleDESC();
            desc.setId(rs.getInt("id"));
            desc.setName(rs.getString("name"));
            desc.setAge(rs.getInt("age"));
            desc.setDesc(rs.getString("description"));
            return desc;
        });

        System.out.println("people_desc table data:");
        for (PeopleDESC desc : resultDesc) {
            System.out.println(desc);
        }

        // Output information after the job is completed.
        System.out.println("Batch Job execution completed.");
    }
}

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.People;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
@SpringBootApplication
@ComponentScan("com.oceanbase.example.batch.writer")
public class AddPeopleWriterTest {

    @Autowired
    private AddPeopleWriter addPeopleWriter;
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Test
    public void testWrite() throws Exception {
        // Insert data into the people table.
        List<People> peopleList = new ArrayList<>();
        People person1 = new People();
        person1.setName("zhangsan");
        person1.setAge(27);
        peopleList.add(person1);
        People person2 = new People();
        person2.setName("lisi");
        person2.setAge(35);
        peopleList.add(person2);
        addPeopleWriter.write(peopleList);

        // Query and output the result.
        List<People> result = jdbcTemplate.query("SELECT * FROM people", (rs, rowNum) -> {
            People person = new People();
            person.setName(rs.getString("name"));
            person.setAge(rs.getInt("age"));
            return person;
        });

        System.out.println("people table data:");
        for (People person : result) {
            System.out.println(person);
        }

        // Output information after the job is completed.
        System.out.println("Batch Job execution completed.");
    }
}

References

For more information about OceanBase Connector/J, see OceanBase JDBC driver.

Applicability

Prerequisites

Note

Procedure

Note

Step 1: Obtain the connection string of OceanBase Database

Note

Step 2: Import the java-oceanbase-springbatch project into IntelliJ IDEA

Note

Step 3: Modify the database connection information in the java-oceanbase-springbatch project

Step 4: Run the java-oceanbase-springbatch project

Project code

Introduction to the pom.xml file

Note

Introduction to the application.properties file

Note

Introduction to the BatchApplication.java file

Introduction to the BatchConfig.java file

Introduction to the People.java file

Introduction to the PeopleDESC.java file

Introduction to the AddPeopleDescProcessor.java file

Introduction to the AddDescPeopleWriter.java file

AddPeopleWriter.java file

BatchConfigTest.java file

AddPeopleDescProcessorTest.java file

File description: AddDescPeopleWriterTest.java

AddPeopleWriterTest.java File Description

Complete code

References

Step 2: Import the `java-oceanbase-springbatch` project into IntelliJ IDEA

Step 3: Modify the database connection information in the `java-oceanbase-springbatch` project

Step 4: Run the `java-oceanbase-springbatch` project