Batch Analyzer - 3.1

Important:

  • Before installing this extension providing Spring Batch and JSR-352 support for JEE, you must uninstall first com.castsoftware.uc.springbatch if present. The two extensions cannot be used simultaneously. Also, migration is not supported between the two.
  • In order to avoid duplication and confusion, when using the Spring Batch extension, any Spring Batch Job and Spring Batch Step objects created by the JEE analyzer will be deleted from the Analysis schema at application level and replaced with corresponding objects created by the Batch extension. You should therefore expect a small impact on existing analysis results in terms of the number of added/deleted objects.

Extension ID

com.castsoftware.springbatch

What’s New?

Please see Batch Analyzer - 3.1 - Release Notes for more information.

Description

In what situation should you install this extension?

The main goal of this extension is to analyze a batch application and enable linking between spring batch objects and between Java objects and Batch objects. If a JEE application contains source code which uses Spring Batch or JSR-352 frameworks and you want to view these objects and also the links with other objects, then you should install this extension.

Features of Spring Batch Framework with xml

Spring Batch is a framework designed to develop robust batch applications. It is not a scheduling framework, but it can work in conjuction with a scheduler (take the example of Quartzexternal link).

Batch procesing can be described as a form of reading in large amounts of data, process it and writing the result out. The architecture of Spring Batch contains three major high-level components: Application, Core, and Infrastructure. The Application contains all batch jobs and custom code written using Spring Batch. The Batch Core contains the core runtime classes necessary to launch and control a batch job. It includes implementations of JobLauncher, Job and Step. Both Application and Core are built on top of a common Infrastructure which contains readers, writers, services (ItemReader, ItemWriter, ItemProcessor).

A Job is an entity which encapsulates an entire batch process. A Job may be configured using either an XML configuration file or a Java-based configuration file. The Spring Batch extension handles the XML configuration file only.

The Spring Batch extension enables linking between:

  • Java objects running Batch Jobs and the corresponding Batch Jobs
  • Batch objects and methods controlling their execution
  • Batch objects and methods defined in batch artifacts

Configuration of a Job

The batch process is encapsulated in a JOB. A JOB consists of multiple STEPS. Each STEP has only one READER-PROCESSOR-WRITER sequence. A JOB is executed by a JobLauncher. Metadata about configured and executed jobs is encapsulated in JobRepository. Each JOB is associated with JobInstances. A JobInstance is defined uniquely by its JobParameters (used to start a batch job). Each run of a JobInstance is referred to as a JobExecution. Each STEP has an individual StepExecution.

job.xml

<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:job id="helloWorldJob">    //helloWorldJob Job consisting of steps,steps having chunk reader, writer, processor
    <batch:step id="step1"> 
        <batch:tasklet>   
            <batch:chunk reader="cvsFileItemReader" writer="xmlItemWriter" processor="itemProcessor" commit-interval="10"> //reader writer and processor dependency   
            </batch:chunk>
        </batch:tasklet>   
    </batch:step>
</batch:job>
...
<bean id="cvsFileItemReader" class="ExampleItemReader" scope="step"/>
<bean id="xmlItemWriter" class="ExampleItemWriter" scope="step"/>
<bean id="itemProcessor" class="ExampleItemProcessor" scope="step"/>

context.xml

<bean id="jobLauncher"  class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> //joblauncher having dependency of JobRepository  
    <property name="jobRepository" ref="jobRepository" /> 
</bean>
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"> 
    <property name="dataSource" ref="dataSource" />        
    <property name="transactionManager" ref="transactionManager" />  
    <property name="databaseType" value="mysql" />
</bean>
<bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />

There may be cases where a JOB is referenced inside a STEP. In this situation, we will look for the referenced JOB, and if it exists, we will add a link to the STEP which invoked it. This is illustrated in the code bellow:

job.xml

<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:step id="step1"> 
    <batch:job ref="helloWorldJob"/>  
</batch:step>
...
<bean id="cvsFileItemReader" class="ExampleItemReader" scope="step"/>
<bean id="xmlItemWriter" class="ExampleItemWriter" scope="step"/>
<bean id="itemProcessor" class="ExampleItemProcessor" scope="step"/>

There are two approaches to build a STEP.

Tasklet based

The execution of the Spring Batch Step is invoked in classes which inherit from Tasklet class. The execute() method is defined inside the class and each call to the Tasklet is wrapped in a Transaction. Tasklet implementors might call a stored procedure, a script, or a simple SQL update statement. We create links between the execute() methods and the STEP. To create a TaskletStep, the ‘ref’ attribute of the <tasklet/> element should reference a bean that defines a Tasklet. No <chunk/> element should be used within the <tasklet/>.

job.xml

<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:job id="helloWorldJob">    //helloWorldJob Job consisting of steps,steps having tasklet   
    <batch:step id="step1"> 
        <batch:tasklet ref="exampleTasklet">
        </batch:tasklet>   
    </batch:step>
</batch:job>
...
<bean id="exampleTasklet" class="ExampleTasklet "scope="step">
    ...
</bean>

ExampleTasklet.java

import org.springframework.batch.core.ExitStatus;
import org.springframework.batch.core.step.tasklet.Tasklet;

public class ExampleTasklet implements Tasklet{
    @Override
    public ExitStatus execute(){
        ...
    }
}

Chunk-oriented processing

Consists reading the data sequentially and creating “chunks” that will be written out within a transaction boundary. Each individual item is read in from an ItemReader, handed to an ItemProcessor, and once the number of items read equals the commit interval (if specified), the entire chunk is written out via the ItemWriter, and then the transaction is committed.

method interface
read()

org.springframework.batch.item.ItemReader

org.springframework.batch.item.ItemStreamReader

process()

org.springframework.batch.item.ItemProcessor

org.springframework.batch.item.support.ScriptItemProcessor

write()

org.springframework.batch.item.ItemWriter

org.springframework.batch.item.ItemStreamWriter

We create links between the STEP and the methods read(), process() and write().

ExampleItemProcessor.java

import org.springframework.batch.item.ItemStreamReader;
import org.springframework.beans.factory.annotation.Autowired;

public class ExampleItemReader implements ItemReader<Class1>{
    @Override
    public Class1 read() throws Exception{
        ...
    }
}

ExampleItemProcessor.java

import org.springframework.batch.item.ItemProcessor;
import org.springframework.beans.factory.annotation.Autowired;

public class ExampleItemProcessor implements ItemProcessor<Class1, Class2>{
    @Override
    public Class2 process(Class1 arg) throws Exception{
        ...
    }
}

ExampleItemProcessor.java

import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;

public class ExampleItemWriter implements ItemWriter<Class1>{
    @Override
    public void write(List<? extends Class1> items) throws Exception{
        ...
    }
}

Running a Job

A container is responsible for instantiating, configuring and assembling the beans. There are 2 types of containers: BeanFactoryexternal link and ApplicationContextexternal link. ApplicationContext is a superset of BeanFactory. You can find hereexternal link the difference between them. ApplicationContext is an interface in the org.springframework.context package and it has several implementations, and the ClassPathXmlApplicationContext is one of these. getBean()external link is a method from the BeanFactory interface responsible for retrieving a bean instance from the Spring container. The getBean() method is most frequently accessed through the ApplicationContext. ApplicationContext provides the JobLauncher to run the JOB*.*

Main.java

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionException;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class Main {
    public static void main(String [] args){
 
    String[] springConfig  =  {"spring/batch/jobs/job.xml" };  //Load the Job.xml file,this Job.xml has dependency on JobLauncher
                                                              // which will load the Repository  
    ApplicationContext context = new ClassPathXmlApplicationContext(springConfig);
    JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher"); //get the JobLauncher object 
    Job job = (Job) context.getBean("helloWorldJob");   //get the Job Object
 
    try {  
        JobExecution execution = jobLauncher.run(job, new JobParameters()); //start the job by calling the run method
        System.out.println("Exit Status : " + execution.getStatus());
    } catch (Exception e) {  
        e.printStackTrace(); 
    } 
    }
}

Listeners in Spring Batch

Spring Batch framework uses listeners to provide progress information on Spring Batch Jobs and Spring Batch Steps. These are called event-driven listeners. The most commonly used listeners are: JobExecutionListener, StepListener and ChunckListener.

Exemple of code for job listener declaration in xml file:

job.xml

<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:job id="helloWorldJob">    //helloWorldJob Job consisting of steps,steps having tasklet   
    <batch:step id="step1"> 
        <batch:tasklet ref="exampleTasklet">
        </batch:tasklet>   
    </batch:step>
    <listeners>
        <listener ref="jobListener"></listener>
    </listeners>
</batch:job>
...
<bean id="exampleTasklet" class="ExampleTasklet "scope="step">
    ...
</bean>
<beans id="jobListener" class="ExampleListenerJob">
    ...
</bean>

Exemple of code for step listener declaration in xml file:

job.xml

<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:job id="helloWorldJob">    //helloWorldJob Job consisting of steps,steps having tasklet   
    <batch:step id="step1"> 
        <batch:tasklet ref="exampleTasklet">
        </batch:tasklet>
        <listeners>
            <listener ref="stepListener"></listener>
        </listeners> 
    </batch:step>
</batch:job>
...
<bean id="exampleTasklet" class="ExampleTasklet "scope="step">
    ...
</bean>
<beans id="stepListener" class="ExampleListenerStep">
    ...
</bean>

In the case of JOBS, the class responsible for supervising the execution is ‘org.springframework.batch.core.JobExecutionListener’. Two methods allow to see the status of the JOB: beforeJob() and afterJob().

ExampleJobExecutionListener.java

import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;

public class ExampleJobExecutionListener implements JobExecutionListener
{
    @Override
    public void beforeJob(JobExecution jobExecution)
    {
        ...
    }
    @Override
    public void afterJob(JobExecution jobExecution)
    {
        ...
    }
}

There might be cases where the classes containing the methods do not inherit from JobExecutionListener class. In this case, the annotations @BeforeJob and @AfterJob are used. Links between these methods and the corresponding JOBS are created with Spring Batch extension.

In the case of the STEPS, the classes responsible for supervising the execution are: ‘org.springframework.batch.core.StepExecutionListener’, ‘org.springframework.batch.core.ItemReadListener’, ‘org.springframework.batch.core.ItemWriteListener’, ‘org.springframework.batch.core.ItemProcessListener’, ‘org.springframework.batch.core.ChunkListener’. The corresponding methods are: ‘afterStep()’, ‘beforeStep()’, ‘beforeWrite()’, ‘afterWrite()’, ‘onWriteError()’, ‘beforeProcess()’, ‘afterProcess()’, ‘onProcessError()’, ‘beforeRead()’, ‘afterRead()’, ‘onReadError()’, ‘beforeChunk()’, ‘afterChunk()’, ‘afterChunkError()’. Here also there are cases where annotations like @BeforeStep and @AfterStep are used.

Links between these methods and the corresponding STEPS are created in the extension for Spring Batch framework.

Moving from one step to another in Spring Batch

The job is a collection of step instances. It combines multiple steps that are logically organized in a flow. In the XML file, tags like ‘split’, ‘flow’ and ‘decision’ are used to organize the flow. Steps can be organized in sequential flow, meaning that the steps are being executed one after another. They can also be organized in conditional flow. In this second case, a step is executed depending on its status (FAILED, STARTED, COMPLETED, …).

In order to decide which step will be executed next, in some cases, a ‘org.springframework.batch.core.job.flow.JobExecutionDecider’ can be used to assist in the decision. The corresponding method is the decide() method. In Spring Batch extension, links are created between the decider step and the decide() method.

To make the difference between the sequential steps and the conditional steps, a property is added to the CAST step object. The name of the property is CAST_Batch_Step.step_type and its value can be ‘split’, ‘flow’ or ‘decision’, depending on the type of tag detected.

Example of a flow when ‘flow’ tag is used.

job.xml

<b:job id="helloWorldJob" job-repository="jobRepository"
        restartable="true">
    ...
    <b:flow id="writeFlowStep">
        <b:step id="writeStartMessageStep" next="writeReturnStep">
            <b:tasklet ref="writeStartMessageTasklet" />
            <b:listeners>
                ...
            </b:listeners>
        </b:step>
        <b:step id="writeReturnStep" next="writeAAAStep">
            <b:tasklet transaction-manager="transactionManager">
                ...
            </b:tasklet>
            <b:listeners>
                <b:listener ref="itemFailureLoggerListener" />
            </b:listeners>
        </b:step>
        <b:step id="writeAAAStep" next="writeEndMessageStep" parent="writeReturnStep">
            <b:tasklet transaction-manager="transactionManager">
                ...
            </b:tasklet>
        </b:step>
        <b:step id="writeEndMessageStep">
            <b:tasklet ref="ecrireFinMessageTasklet" />
            <b:listeners>
                ...
            </b:listeners>
        </b:step>
    </b:flow>
    ...
</b:job>

The illustration of step chain:

The new property of the spring batch step:

Example of a flow when ‘split’ tag is used.

job.xml

<b:job id="helloWorldJob" job-repository="jobRepository"
        restartable="true">    
    <b:split id="splitStep"
            task-executor="taskExecutor">
        <b:flow>
            <b:step id="step6"
                next="step7">
                <b:tasklet transaction-manager="transactionManager">
                    <b:chunk
                        commit-interval="${...}"
                        processor="creerFichierProcessor"
                        reader="creerFichierReader"
                        writer="creerFichierWriter" />
                </b:tasklet>
                <b:listeners>
                    <b:listener ref="itemListener" />
                    <b:listener ref="stepListener" />
                    <b:listener ref="chunkListener" />
                </b:listeners>
            </b:step>
            <b:step id="step7">
                <b:tasklet ref="fileDate" />
                <b:listeners>
                    <b:listener ref="itemListener" />
                </b:listeners>
            </b:step>             
        </b:flow>
    </b:split>
</b:job>

The illustration of step chain:

The new property of the spring batch step:

Example of a flow when ‘decision’ tag is used.

job.xml

<b:job id="helloWorldJob" job-repository="jobRepository"
        restartable="true">    
    <b:step id="step1" next="decision1">
        <b:tasklet ref="appelServiceSeuilRejetTasklet"/>
        <b:listeners>
            <b:listener ref="itemFailureLoggerListener" />
            <b:listener ref="sisoStepListener" />
        </b:listeners>
    </b:step>

    <b:decision decider="decider1" id="decision1">
        <b:next on="FAILED" to="step2" />
        <b:next on="COMPLETED" to="step3" />
    </b:decision>

    <b:step id="step3">
        <b:tasklet ref="VerifierListHabilitationsEmetteurTasklet" />
        <b:listeners>
            <b:listener ref="itemFailureLoggerListener" />
            <b:listener ref="sisoStepListener" />
        </b:listeners>
    </b:step>

    <b:decision id="step2"
        decider="genererBilanDecider">
        <b:next on="TRUE" to="step4" />
        <b:end on="COMPLETED"/>
    </b:decision>
        
    <b:step id="step4">
        <b:tasklet ref="debutFichierRetourTasklet" />
        <b:listeners>
            <b:listener ref="itemFailureLoggerListener" />
        </b:listeners>
    </b:step>
</b:job>

beans.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns:b="http://www.springframework.org/schema/batch"
    xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/batch 
                        http://www.springframework.org/schema/batch/spring-batch.xsd
                        http://www.springframework.org/schema/beans 
                        http://www.springframework.org/schema/beans/spring-beans.xsd">
    ...
    <bean id="decider1"
        class="Decider">
    </bean>
    ...
</beans>

Decider.java

import org.springframework.batch.core.job.flow.JobExecutionDecider;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.StepExecution;

public class Decider implements JobExecutionDecider {
    @Override
    public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) {
    ...
    }
}

The illustration of step chain:

The new property of the spring batch step:

Features of JSR-352 Framework with xml

JSR-352 is a framework similar to Spring Batch. Just as mentioned in the documentation above, it’s functionalities are based on jobs, steps, readers, writers, processors, listeners.

The first difference between the two frameworks is the job execution, as illustrated in the code example below. The interface javax.batch.operations.JobOperator allows the execution of the job. This implementation is loaded via thejavax.batch.runtime.BatchRuntime interface.

Main.java

import javax.batch.operations.JobOperator;
import javax.batch.runtime.BatchRuntime;
import java.util.Properties;

public class Main {
    public static void main(String [] args){
        JobOperator operator = BatchRuntime.getJobOperator();
        jobOperator.start("helloWorldJob", new Properties()); 
}

Another difference is the structure of the xml file and the tag names. The “tasklet” tag is called “batchlet” in the JSR-352 framework. As for readers, writers and processors, which were attibutes of “chunk” tag in Spring Batch framework, they become individual tags under the “chunk” tag.

job.xml

<job id="helloWorldJob">    //helloWorldJob Job consisting of steps,steps having chunk reader, writer, processor
    <step id="step1"> 
        <chunk>   
            <reader="cvsFileItemReader">
            <writer="xmlItemWriter">
            <processor="itemProcessor" commit-interval="10">   
        </chunk>   
    </step>
</job>

The classes we support in JSR-352 framework are:

method interface
readItem()

javax.batch.api.chunk.AbstractItemReader

processItem() javax.batch.api.chunk.ItemProcessor
writeItem() javax.batch.api.chunk.AbstractItemWriter
process(), stop()

javax.batch.api.AbstractBatchlet

javax.batch.api.Batchlet

beforeJob(), afterJob() javax.batch.api.listener.JobListener
beforeStep(), afterStep() javax.batch.api.listener.StepListener

Links between these methods and the corresponding JOBS and STEPS are created in the extension for JSR-352 framework.

Features of Spring Batch Framework in Java

Supported APIs:

Job APIs Step APIs
org.springframework.batch.core.job.builder.SimpleJobBuilder.build org.springframework.batch.core.step.builder.SimpleStepBuilder.build
org.springframework.batch.core.job.builder.FlowJobBuilder.build org.springframework.batch.core.step.builder.FlowStepBuilder.build
org.springframework.batch.core.job.builder.JobFlowBuilder.build org.springframework.batch.core.step.builder.JobStepBuilder.build

org.springframework.batch.core.step.builder.AbstractTaskletStepBuilder.build

org.springframework.batch.core.step.builder.PartitionStepBuilder.build
org.springframework.batch.core.job.builder.JobBuilder.flow org.springframework.batch.core.step.builder.StepBuilder.tasklet

org.springframework.batch.core.step.builder.StepBuilder.partitioner
org.springframework.batch.core.job.builder.JobBuilder.start org.springframework.batch.core.step.builder.SimpleStepBuilder.reader
org.springframework.batch.core.job.builder.JobBuilderHelper.listener org.springframework.batch.core.step.builder.SimpleStepBuilder.processor

org.springframework.batch.core.step.builder.SimpleStepBuilder.writer
org.springframework.batch.core.job.builder.FlowBuilder.next
org.springframework.batch.core.job.builder.SimpleJobBuilder.next
org.springframework.batch.core.job.builder.FlowBuilder.TransitionBuilder.to
org.springframework.batch.core.configuration.annotation.JobBuilderFactory.get org.springframework.batch.core.configuration.annotation.StepBuilderFactory.get

Configuration of a Job in Java

BatchConfiguration.java

Job job = jobBuilderFactory
        .get("financialsJob")
        .incrementer(new RunIdIncrementer())
        .flow(statementsSteps)
        .end()
        .build();

Configuration of a Step in Java

BatchConfiguration.java

@Bean
  public Step statementsSteps(ItemReader<String> statementsReader,
      ItemWriter<Map<Integer, List<StatementHeader>>> changeAlertsWriter,
      ItemProcessor<? super String, ? extends Map<Integer, List<StatementHeader>>> statementsProcessor) {
    return stepBuilderFactory
        .get("statementsSteps")
        .chunk(1)
        .reader(statementsReader)
        .processor(statementsProcessor)
        .writer(changeAlertsWriter).build();

Define step sequence in Java

SpringBatchConfiguration.java

...
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
...


@Configuration
@EnableBatchProcessing
public class SpringBatchConfiguration {

    private static Logger LOGGER = LoggerFactory.getLogger(SpringBatchConfiguration.class);

    private static final String[] TOKENS = { "bookname", "bookauthor", "bookformat", "isbn", "publishyear" };

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    ...

    @Bean
    public Step step1(ItemReader<BookRecord> csvItemReader, ItemWriter<Book> jsonItemWriter) throws IOException {
        return stepBuilderFactory
          .get("step1")
          .<BookRecord, Book> chunk(3)
          .reader(csvItemReader)
          .processor(bookItemProcessor())
          .writer(jsonItemWriter)
          .build();
    }

    @Bean
    public Step step2(ItemReader<BookRecord> csvItemReader, ItemWriter<BookDetails> listItemWriter) {
        return stepBuilderFactory
          .get("step2")
          .<BookRecord, BookDetails> chunk(3)
          .reader(csvItemReader)
          .processor(bookDetailsItemProcessor())
          .writer(listItemWriter)
          .build();
    }

    @Bean(name = "transformBooksRecords")
    public Job transformBookRecords(Step step1, Step step2) throws IOException {
        return jobBuilderFactory
          .get("transformBooksRecords")
          .flow(step1)
          .next(step2)
          .end()
          .build();
    }

}

Listeners with Spring Batch in Java

SpringBatchConfig.java

import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;

import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;

import javax.annotation.Resource;
import org.springframework.beans.factory.annotation.Qualifier;
import com.job.listener.methods.JobTRExecutionListener;


public class SpringBatchConfig {
    
    @Autowired
    private JobBuilderFactory jobs;

    @Autowired
    private StepBuilderFactory steps;

    @Value("input/record.csv")
    private Resource inputCsv;

    @Value("file:xml/output.xml")
    private Resource outputXml;
    
    private JobTRExecutionListener listener;

    ...

    @Bean
    protected Step step1(ItemReader<Transaction> reader,
      ItemProcessor<Transaction, Transaction> processor,
      ItemWriter<Transaction> writer) {
        return steps.get("step1").chunk(10)
          .reader(reader).processor(processor).writer(writer).build();
    }

    @Bean(name = "firstBatchJob")
    public Job job(@Qualifier("step1") Step step1) {
        return jobs.get("firstBatchJob").listener(listener).start(step1).build();
    }
}

Tasklets with Spring Batch in Java

SpringBatchConfigFlow .java

import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;

import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;

import javax.annotation.Resource;
import org.springframework.beans.factory.annotation.Qualifier;

import com.jobs.scheduler.tasklets.AcquisitionTasklet;


public class SpringBatchConfigFlow {
    @Autowired
    private JobBuilderFactory jobs;

    @Autowired
    private StepBuilderFactory steps;
    
    private AcquisitionTasklet tasklet;

    @Bean
    protected Step step2(ItemReader<Transaction> reader,
      ItemProcessor<Transaction, Transaction> processor,
      ItemWriter<Transaction> writer) {
        return steps.get("step2").tasklet(tasklet).build();
    }
    ...
}

AcquisitionTasklet .java

package com.jobs.scheduler.tasklets;

import org.springframework.batch.core.ExitStatus;
import org.springframework.batch.core.StepContribution;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.http.HttpStatus;

import com.sncf.eva.tr.jobs.rest.client.JobsClientRest;

public class AcquisitionTasklet implements Tasklet {
    /
     * Logger
     */
    public static final org.slf4j.Logger LOG = org.slf4j.LoggerFactory.getLogger(AcquisitionTasklet.class);

    public static final String PARAM_TASKLET_DATE = "date";

    private JobsClientRest jobsClientRest;

    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
        LOG.info("Execution de la tasklet acquisition pour les paramètres : Context - {}",
            getParam(chunkContext, PARAM_TASKLET_DATE));

        contribution.incrementFilterCount(1);

        executeServices(contribution);

        return RepeatStatus.FINISHED;
    }
    ...

Partitioner with Spring Batch in Java

The partitioner allows parallel execution of steps.

SpringbatchPartitionConfig .java

import com.baeldung.batch.model.Transaction;
import com.baeldung.batch.service.RecordFieldSetMapper;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
...

@Configuration
@EnableBatchProcessing
public class SpringbatchPartitionConfig {

    @Autowired
    private JobBuilderFactory jobs;

    @Autowired
    private StepBuilderFactory steps;

    @Bean(name = "partitionerJob")
    public Job partitionerJob() throws UnexpectedInputException, MalformedURLException, ParseException {
        return jobs.get("partitionerJob")
          .start(partitionStep())
          .build();
    }

    @Bean
    public Step partitionStep() throws UnexpectedInputException, MalformedURLException, ParseException {
        return steps.get("partitionStep")
          .partitioner("slaveStep", partitioner())
          .step(slaveStep())
          .taskExecutor(taskExecutor())
          .build();
    }

    @Bean
    public Step slaveStep() throws UnexpectedInputException, MalformedURLException, ParseException {
        return steps.get("slaveStep")
          .<Transaction, Transaction>chunk(1)
          .reader(itemReader(null))
          .writer(itemWriter(marshaller(), null))
          .build();
    }
    ...
}

Access to databases in xml and Java

Item beans provide access to databases using cursor-based and paging-based implementations in Java and XML.

Supported APIs in Java

Java API Query Object created
org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider.setSelectClause CAST_Batch_SQL_Query
org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider.setFromClause
org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider.setWhereClause
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.setSelectClause CAST_Batch_SQL_Query
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.setFromClause
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.setWhereClause
org.springframework.batch.item.database.builder.JdbcPagingItemReaderBuilder.build CAST_Batch_JDBC_Query
org.springframework.batch.item.database.builder.JdbcPagingItemReaderBuilder.selectClause
org.springframework.batch.item.database.builder.JdbcPagingItemReaderBuilder.fromClause
org.springframework.batch.item.database.builder.JdbcPagingItemReaderBuilder.whereClause
org.springframework.batch.item.database.JdbcBatchItemWriter.setSql CAST_Batch_JDBC_Query
org.springframework.batch.item.database.builder.JdbcBatchItemWriterBuilder.sql
org.springframework.batch.item.database.JdbcCursorItemReader.setSql CAST_Batch_JDBC_Query
org.springframework.batch.item.database.builder.JdbcCursorItemReaderBuilder.sql
org.springframework.batch.item.database.JpaCursorItemReader.setQueryString CAST_Batch_JPA_SQLQuery
org.springframework.batch.item.database.HibernateCursorItemReader.setQueryString
org.springframework.batch.item.database.JpaPagingItemReader.setQueryString
org.springframework.batch.item.database.HibernatePagingItemReader.setQueryString
org.springframework.batch.item.database.builder.JpaPagingItemReaderBuilder.queryString
org.springframework.batch.item.database.StoredProcedureItemReader.setProcedureName CAST_Batch_JDBC_Query
org.springframework.batch.item.database.orm.JpaNativeQueryProvider.setSqlQuery CAST_Batch_SQL_Query
org.springframework.batch.item.database.orm.HibernateNativeQueryProvider.setSqlQuery

Supported APIs in XML

XML TAG Query Object created
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.selectClause CAST_Batch_SQL_Query
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.fromClause
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.whereClause
org.springframework.batch.item.database.JdbcCursorItemReader sql CAST_Batch_JDBC_Query
org.springframework.batch.item.database.JdbcBatchItemWriter sql
org.springframework.batch.item.database.JpaPagingItemReader queryString CAST_Batch_JPA_SQLQuery
org.springframework.batch.item.database.HibernateCursorItemReader queryString
org.springframework.batch.item.database.StoredProcedureItemReader procedureName CAST_Batch_JDBC_Query

Examples

Support of org.springframework.batch.item.database.JpaPagingItemReader.queryString

import org.springframework.batch.item.database.JpaPagingItemReader;
import org.springframework.batch.item.database.builder.JpaPagingItemReaderBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
 
import javax.persistence.EntityManagerFactory;
 
@Configuration
public class EnfantReaderConfig {
 
    @Value("${batch.remplacement-reco.reader-page-size:1000}")
    private int READER_PAGE_SIZE;
 
    @Bean
    public JpaPagingItemReader<Enfant> enfantReader(EntityManagerFactory entityManagerFactory) {
        return new JpaPagingItemReaderBuilder<Enfant>()
                .name("enfantReader")
                .entityManagerFactory(entityManagerFactory)
                .queryString("SELECT enfant FROM Enfant enfant")
                .pageSize(READER_PAGE_SIZE)
                .build();
    }
 
}

Support of org.springframework.batch.item.database.StoredProcedureItemReader.setProcedureName

package com.example.batch.config;
 
import javax.sql.DataSource;
import org.springframework.batch.item.database.StoredProcedureItemReader;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
 
@Configuration
public class BatchConfig {
 
    @Bean
    public StoredProcedureItemReader<CustomerCredit> reader(DataSource dataSource) {
        StoredProcedureItemReader<CustomerCredit> reader = new StoredProcedureItemReader<>();
 
        reader.setDataSource(dataSource);
        reader.setProcedureName("sp_customer_credit");
        reader.setRowMapper(new CustomerCreditRowMapper());
 
        return reader;
    }
}

Support of org.springframework.batch.item.database.StoredProcedureItemReader in xml for procedureName property tag

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:batch="http://www.springframework.org/schema/batch"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans.xsd
           http://www.springframework.org/schema/batch
           http://www.springframework.org/schema/batch/spring-batch.xsd">
 
    <batch:job id="customerCreditJob">
        <batch:step id="step1">
            <batch:tasklet>
                <batch:chunk reader="reader" writer="writer" commit-interval="10"/>
            </batch:tasklet>
        </batch:step>
    </batch:job>
 
    <bean id="writer" class="org.springframework.batch.item.support.ListItemWriter"/>
 
</beans>
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans.xsd">

 
    <bean id="reader" class="org.springframework.batch.item.database.StoredProcedureItemReader">
        <property name="dataSource" ref="dataSource"/>
        <property name="procedureName" value="sp_customer_credit"/>
        <property name="rowMapper">
            <bean class="com.example.batch.config.CustomerCreditRowMapper"/>
        </property>
    </bean>
 
</beans>

Support of org.springframework.batch.item.database.JpaPagingItemReader in xml for queryString property tag

<batch:job id="records" incrementer="jobIncrementer">
        <!--Step de création des Fiches de raccordement -->
        <batch:step id="stepRecords" next="stepRecordsUpdate">
            <tasklet>
                <chunk 
                    reader="recordsCreateReader" 
                    processor="recordsProcessor"
                    writer="recordsCreateWriter" 
                    commit-interval="2" />
                <batch:fail on="FAILED" />
            </tasklet>
        </batch:step>
</batch:job>
<beans:bean id="recordsCreateReader" class="org.springframework.batch.item.database.JpaPagingItemReader">
	<beans:property name="entityManagerFactory" ref="entityManagerFactory" />
	<beans:property name="queryString" value="SELECT DataRecord struct FROM FileRecords file INNER JOIN file.structureActive struct WHERE file.currentState = fileRecordState.SENT" />
</beans:bean>

Support of org.springframework.batch.item.database.builder.JdbcPagingItemReaderBuilder setSelect/setFrom/setWhereClause

	public SqlPagingQueryProviderFactoryBean queryProvider() {
		SqlPagingQueryProviderFactoryBean provider = new SqlPagingQueryProviderFactoryBean();

		provider.setSelectClause("select id, name, credit");
		provider.setFromClause("from customer_credit");
		provider.setWhereClause("where status=:status");
		provider.setSortKey("id");

		return provider;
	}

Support of org.springframework.batch.item.database.JdbcCursorItemReader in xml for sql property tag

<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:job id="helloWorldJob">    //helloWorldJob Job consisting of steps,steps having chunk reader, writer, processor
    <batch:step id="stepInformation"> 
        <batch:tasklet>   
            <batch:chunk reader="infoItemReader" writer="infoItemWriter" processor="itemProcessor" commit-interval="10"> //reader writer and processor dependency   
            </batch:chunk>
        </batch:tasklet>   
    </batch:step>
</batch:job>
...
<bean id="infoItemReader" class="org.springframework.batch.item.database.JdbcCursorItemReader" scope="step">
    <property name="sql"
            value="SELECT * FROM INFORMATIONS"/>
</bean>

Compatibility

Core release Operating System Supported
v3/8.4.x Microsoft Windows / Linux
v2/8.3.x Microsoft Windows

Dependencies with other extensions

Some CAST extensions require the presence of other CAST extensions in order to function correctly. The Spring Batch extension requires that the following other CAST extensions are also installed:

  • com.castsoftware.java.hibernate (≥ 1.0.7)
  • com.castsoftware.internal.platform (internal technical extension)

What results can you expect?

Once the analysis/snapshot generation has completed, you can visualize the results in the usual manner:

  • Spring Batch framework

  • JSR-352 framework

Objects

Icon Object Type
Batch Job
Batch Step
Batch SQL Query
Batch JDBC Query
Batch JPQL Query

In order to avoid duplication and confusion, when using the Spring Batch extension, any Spring Batch Job and Spring Batch Step objects created by the JEE analyzer will be deleted from the Analysis schema at application level.

The following call links are added:

  • Between Batch Jobs and Batch Steps
  • Between Batch Jobs and beforeJob() and afterJob() methods defined in JobExecutionListener classes
  • Between the methods calling jobs and Batch Jobs
  • Between Batch Steps and other Batch Steps
  • Between Batch Steps and execute() methods defined in Tasklet classes
  • Between Batch Steps and process() methods defined in ItemProcessor, ItemScriptItemProcessor classes
  • Between Batch Steps and read(), open(), update(), close() methods defined in ItemReader and ItemStreamReader classes
  • Between Batch Steps and write(), open(), update(), close() methods defined in ItemWriter and ItemStreamWriter classes
  • Between Batch Steps and beforeStep(), afterStep(), beforeWrite(), afterWrite(), onWriteError(), beforeProcess(), afterProcess(), onProcessError(), beforeRead(), afterRead(), onReadError(), beforeChunk(), afterChunk(), afterChunkError() methods defined in StepExecutionListener, ItemWriteListener, ItemProcessListener, ItemReadListenerChunkListener classes
  • Between Batch Steps and Spring Beans if beans have sql properties and Spring Beans are linked to tables

Important information

  • When using the Spring Batch extension, any Spring Batch Job and Spring Batch Step objects created by the JEE analyzer will be deleted from the Analysis schema at application level.
  • Support of Java Beans containing SQL queries in Spring Batch framework : links between Batch Step and Spring Bean containing queries are added if Spring Bean are linked to tables.