Batch Analyzer - 3.1
Important:
- Before installing this extension providing Spring Batch and JSR-352 support for JEE, you must uninstall first com.castsoftware.uc.springbatch if present. The two extensions cannot be used simultaneously. Also, migration is not supported between the two.
- In order to avoid duplication and confusion, when using the Spring Batch extension, any Spring Batch Job and Spring Batch Step objects created by the JEE analyzer will be deleted from the Analysis schema at application level and replaced with corresponding objects created by the Batch extension. You should therefore expect a small impact on existing analysis results in terms of the number of added/deleted objects.
Extension ID
com.castsoftware.springbatch
What’s New?
Please see Batch Analyzer - 3.1 - Release Notes for more information.
Description
In what situation should you install this extension?
The main goal of this extension is to analyze a batch application and enable linking between spring batch objects and between Java objects and Batch objects. If a JEE application contains source code which uses Spring Batch or JSR-352 frameworks and you want to view these objects and also the links with other objects, then you should install this extension.
Features of Spring Batch Framework with xml
Spring Batch is a framework designed to develop robust batch applications. It is not a scheduling framework, but it can work in conjuction with a scheduler (take the example of Quartz ).
Batch procesing can be described as a form of reading in large amounts of data, process it and writing the result out. The architecture of Spring Batch contains three major high-level components: Application, Core, and Infrastructure. The Application contains all batch jobs and custom code written using Spring Batch. The Batch Core contains the core runtime classes necessary to launch and control a batch job. It includes implementations of JobLauncher, Job and Step. Both Application and Core are built on top of a common Infrastructure which contains readers, writers, services (ItemReader, ItemWriter, ItemProcessor).
A Job is an entity which encapsulates an entire batch process. A Job may be configured using either an XML configuration file or a Java-based configuration file. The Spring Batch extension handles the XML configuration file only.
The Spring Batch extension enables linking between:
- Java objects running Batch Jobs and the corresponding Batch Jobs
- Batch objects and methods controlling their execution
- Batch objects and methods defined in batch artifacts
Configuration of a Job
The batch process is encapsulated in a JOB. A JOB consists of multiple STEPS. Each STEP has only one READER-PROCESSOR-WRITER sequence. A JOB is executed by a JobLauncher. Metadata about configured and executed jobs is encapsulated in JobRepository. Each JOB is associated with JobInstances. A JobInstance is defined uniquely by its JobParameters (used to start a batch job). Each run of a JobInstance is referred to as a JobExecution. Each STEP has an individual StepExecution.
job.xml
<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:job id="helloWorldJob"> //helloWorldJob Job consisting of steps,steps having chunk reader, writer, processor
<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="cvsFileItemReader" writer="xmlItemWriter" processor="itemProcessor" commit-interval="10"> //reader writer and processor dependency
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
...
<bean id="cvsFileItemReader" class="ExampleItemReader" scope="step"/>
<bean id="xmlItemWriter" class="ExampleItemWriter" scope="step"/>
<bean id="itemProcessor" class="ExampleItemProcessor" scope="step"/>
context.xml
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> //joblauncher having dependency of JobRepository
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseType" value="mysql" />
</bean>
<bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
There may be cases where a JOB is referenced inside a STEP. In this situation, we will look for the referenced JOB, and if it exists, we will add a link to the STEP which invoked it. This is illustrated in the code bellow:
job.xml
<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:step id="step1">
<batch:job ref="helloWorldJob"/>
</batch:step>
...
<bean id="cvsFileItemReader" class="ExampleItemReader" scope="step"/>
<bean id="xmlItemWriter" class="ExampleItemWriter" scope="step"/>
<bean id="itemProcessor" class="ExampleItemProcessor" scope="step"/>
There are two approaches to build a STEP.
Tasklet based
The execution of the Spring Batch Step is invoked in classes which inherit from Tasklet class. The execute() method is defined inside the class and each call to the Tasklet is wrapped in a Transaction. Tasklet implementors might call a stored procedure, a script, or a simple SQL update statement. We create links between the execute() methods and the STEP. To create a TaskletStep, the ‘ref’ attribute of the <tasklet/> element should reference a bean that defines a Tasklet. No <chunk/> element should be used within the <tasklet/>.
job.xml
<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:job id="helloWorldJob"> //helloWorldJob Job consisting of steps,steps having tasklet
<batch:step id="step1">
<batch:tasklet ref="exampleTasklet">
</batch:tasklet>
</batch:step>
</batch:job>
...
<bean id="exampleTasklet" class="ExampleTasklet "scope="step">
...
</bean>
ExampleTasklet.java
import org.springframework.batch.core.ExitStatus;
import org.springframework.batch.core.step.tasklet.Tasklet;
public class ExampleTasklet implements Tasklet{
@Override
public ExitStatus execute(){
...
}
}
Chunk-oriented processing
Consists reading the data sequentially and creating “chunks” that will be written out within a transaction boundary. Each individual item is read in from an ItemReader, handed to an ItemProcessor, and once the number of items read equals the commit interval (if specified), the entire chunk is written out via the ItemWriter, and then the transaction is committed.
method | interface |
---|---|
read() | org.springframework.batch.item.ItemReader org.springframework.batch.item.ItemStreamReader |
process() | org.springframework.batch.item.ItemProcessor org.springframework.batch.item.support.ScriptItemProcessor |
write() | org.springframework.batch.item.ItemWriter org.springframework.batch.item.ItemStreamWriter |
We create links between the STEP and the methods read(), process() and write().
ExampleItemProcessor.java
import org.springframework.batch.item.ItemStreamReader;
import org.springframework.beans.factory.annotation.Autowired;
public class ExampleItemReader implements ItemReader<Class1>{
@Override
public Class1 read() throws Exception{
...
}
}
ExampleItemProcessor.java
import org.springframework.batch.item.ItemProcessor;
import org.springframework.beans.factory.annotation.Autowired;
public class ExampleItemProcessor implements ItemProcessor<Class1, Class2>{
@Override
public Class2 process(Class1 arg) throws Exception{
...
}
}
ExampleItemProcessor.java
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
public class ExampleItemWriter implements ItemWriter<Class1>{
@Override
public void write(List<? extends Class1> items) throws Exception{
...
}
}
Running a Job
A container is responsible for instantiating, configuring and assembling the beans. There are 2 types of containers: BeanFactory and ApplicationContext . ApplicationContext is a superset of BeanFactory. You can find here the difference between them. ApplicationContext is an interface in the org.springframework.context package and it has several implementations, and the ClassPathXmlApplicationContext is one of these. getBean() is a method from the BeanFactory interface responsible for retrieving a bean instance from the Spring container. The getBean() method is most frequently accessed through the ApplicationContext. ApplicationContext provides the JobLauncher to run the JOB*.*
Main.java
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionException;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
public class Main {
public static void main(String [] args){
String[] springConfig = {"spring/batch/jobs/job.xml" }; //Load the Job.xml file,this Job.xml has dependency on JobLauncher
// which will load the Repository
ApplicationContext context = new ClassPathXmlApplicationContext(springConfig);
JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher"); //get the JobLauncher object
Job job = (Job) context.getBean("helloWorldJob"); //get the Job Object
try {
JobExecution execution = jobLauncher.run(job, new JobParameters()); //start the job by calling the run method
System.out.println("Exit Status : " + execution.getStatus());
} catch (Exception e) {
e.printStackTrace();
}
}
}
Listeners in Spring Batch
Spring Batch framework uses listeners to provide progress information on Spring Batch Jobs and Spring Batch Steps. These are called event-driven listeners. The most commonly used listeners are: JobExecutionListener, StepListener and ChunckListener.
Exemple of code for job listener declaration in xml file:
job.xml
<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:job id="helloWorldJob"> //helloWorldJob Job consisting of steps,steps having tasklet
<batch:step id="step1">
<batch:tasklet ref="exampleTasklet">
</batch:tasklet>
</batch:step>
<listeners>
<listener ref="jobListener"></listener>
</listeners>
</batch:job>
...
<bean id="exampleTasklet" class="ExampleTasklet "scope="step">
...
</bean>
<beans id="jobListener" class="ExampleListenerJob">
...
</bean>
Exemple of code for step listener declaration in xml file:
job.xml
<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:job id="helloWorldJob"> //helloWorldJob Job consisting of steps,steps having tasklet
<batch:step id="step1">
<batch:tasklet ref="exampleTasklet">
</batch:tasklet>
<listeners>
<listener ref="stepListener"></listener>
</listeners>
</batch:step>
</batch:job>
...
<bean id="exampleTasklet" class="ExampleTasklet "scope="step">
...
</bean>
<beans id="stepListener" class="ExampleListenerStep">
...
</bean>
In the case of JOBS, the class responsible for supervising the execution is ‘org.springframework.batch.core.JobExecutionListener’. Two methods allow to see the status of the JOB: beforeJob() and afterJob().
ExampleJobExecutionListener.java
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;
public class ExampleJobExecutionListener implements JobExecutionListener
{
@Override
public void beforeJob(JobExecution jobExecution)
{
...
}
@Override
public void afterJob(JobExecution jobExecution)
{
...
}
}
There might be cases where the classes containing the methods do not inherit from JobExecutionListener class. In this case, the annotations @BeforeJob and @AfterJob are used. Links between these methods and the corresponding JOBS are created with Spring Batch extension.
In the case of the STEPS, the classes responsible for supervising the execution are: ‘org.springframework.batch.core.StepExecutionListener’, ‘org.springframework.batch.core.ItemReadListener’, ‘org.springframework.batch.core.ItemWriteListener’, ‘org.springframework.batch.core.ItemProcessListener’, ‘org.springframework.batch.core.ChunkListener’. The corresponding methods are: ‘afterStep()’, ‘beforeStep()’, ‘beforeWrite()’, ‘afterWrite()’, ‘onWriteError()’, ‘beforeProcess()’, ‘afterProcess()’, ‘onProcessError()’, ‘beforeRead()’, ‘afterRead()’, ‘onReadError()’, ‘beforeChunk()’, ‘afterChunk()’, ‘afterChunkError()’. Here also there are cases where annotations like @BeforeStep and @AfterStep are used.
Links between these methods and the corresponding STEPS are created in the extension for Spring Batch framework.
Moving from one step to another in Spring Batch
The job is a collection of step instances. It combines multiple steps that are logically organized in a flow. In the XML file, tags like ‘split’, ‘flow’ and ‘decision’ are used to organize the flow. Steps can be organized in sequential flow, meaning that the steps are being executed one after another. They can also be organized in conditional flow. In this second case, a step is executed depending on its status (FAILED, STARTED, COMPLETED, …).
In order to decide which step will be executed next, in some cases, a ‘org.springframework.batch.core.job.flow.JobExecutionDecider’ can be used to assist in the decision. The corresponding method is the decide() method. In Spring Batch extension, links are created between the decider step and the decide() method.
To make the difference between the sequential steps and the conditional steps, a property is added to the CAST step object. The name of the property is CAST_Batch_Step.step_type and its value can be ‘split’, ‘flow’ or ‘decision’, depending on the type of tag detected.
Example of a flow when ‘flow’ tag is used.
job.xml
<b:job id="helloWorldJob" job-repository="jobRepository"
restartable="true">
...
<b:flow id="writeFlowStep">
<b:step id="writeStartMessageStep" next="writeReturnStep">
<b:tasklet ref="writeStartMessageTasklet" />
<b:listeners>
...
</b:listeners>
</b:step>
<b:step id="writeReturnStep" next="writeAAAStep">
<b:tasklet transaction-manager="transactionManager">
...
</b:tasklet>
<b:listeners>
<b:listener ref="itemFailureLoggerListener" />
</b:listeners>
</b:step>
<b:step id="writeAAAStep" next="writeEndMessageStep" parent="writeReturnStep">
<b:tasklet transaction-manager="transactionManager">
...
</b:tasklet>
</b:step>
<b:step id="writeEndMessageStep">
<b:tasklet ref="ecrireFinMessageTasklet" />
<b:listeners>
...
</b:listeners>
</b:step>
</b:flow>
...
</b:job>
The illustration of step chain:
The new property of the spring batch step:
Example of a flow when ‘split’ tag is used.
job.xml
<b:job id="helloWorldJob" job-repository="jobRepository"
restartable="true">
<b:split id="splitStep"
task-executor="taskExecutor">
<b:flow>
<b:step id="step6"
next="step7">
<b:tasklet transaction-manager="transactionManager">
<b:chunk
commit-interval="${...}"
processor="creerFichierProcessor"
reader="creerFichierReader"
writer="creerFichierWriter" />
</b:tasklet>
<b:listeners>
<b:listener ref="itemListener" />
<b:listener ref="stepListener" />
<b:listener ref="chunkListener" />
</b:listeners>
</b:step>
<b:step id="step7">
<b:tasklet ref="fileDate" />
<b:listeners>
<b:listener ref="itemListener" />
</b:listeners>
</b:step>
</b:flow>
</b:split>
</b:job>
The illustration of step chain:
The new property of the spring batch step:
Example of a flow when ‘decision’ tag is used.
job.xml
<b:job id="helloWorldJob" job-repository="jobRepository"
restartable="true">
<b:step id="step1" next="decision1">
<b:tasklet ref="appelServiceSeuilRejetTasklet"/>
<b:listeners>
<b:listener ref="itemFailureLoggerListener" />
<b:listener ref="sisoStepListener" />
</b:listeners>
</b:step>
<b:decision decider="decider1" id="decision1">
<b:next on="FAILED" to="step2" />
<b:next on="COMPLETED" to="step3" />
</b:decision>
<b:step id="step3">
<b:tasklet ref="VerifierListHabilitationsEmetteurTasklet" />
<b:listeners>
<b:listener ref="itemFailureLoggerListener" />
<b:listener ref="sisoStepListener" />
</b:listeners>
</b:step>
<b:decision id="step2"
decider="genererBilanDecider">
<b:next on="TRUE" to="step4" />
<b:end on="COMPLETED"/>
</b:decision>
<b:step id="step4">
<b:tasklet ref="debutFichierRetourTasklet" />
<b:listeners>
<b:listener ref="itemFailureLoggerListener" />
</b:listeners>
</b:step>
</b:job>
beans.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns:b="http://www.springframework.org/schema/batch"
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/batch
http://www.springframework.org/schema/batch/spring-batch.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
...
<bean id="decider1"
class="Decider">
</bean>
...
</beans>
Decider.java
import org.springframework.batch.core.job.flow.JobExecutionDecider;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.StepExecution;
public class Decider implements JobExecutionDecider {
@Override
public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) {
...
}
}
The illustration of step chain:
The new property of the spring batch step:
Features of JSR-352 Framework with xml
JSR-352 is a framework similar to Spring Batch. Just as mentioned in the documentation above, it’s functionalities are based on jobs, steps, readers, writers, processors, listeners.
The first difference between the two frameworks is the job execution, as
illustrated in the code example below. The interface
javax.batch.operations.JobOperator
allows the execution of the job.
This implementation is loaded via thejavax.batch.runtime.BatchRuntime
interface.
Main.java
import javax.batch.operations.JobOperator;
import javax.batch.runtime.BatchRuntime;
import java.util.Properties;
public class Main {
public static void main(String [] args){
JobOperator operator = BatchRuntime.getJobOperator();
jobOperator.start("helloWorldJob", new Properties());
}
Another difference is the structure of the xml file and the tag names. The “tasklet” tag is called “batchlet” in the JSR-352 framework. As for readers, writers and processors, which were attibutes of “chunk” tag in Spring Batch framework, they become individual tags under the “chunk” tag.
job.xml
<job id="helloWorldJob"> //helloWorldJob Job consisting of steps,steps having chunk reader, writer, processor
<step id="step1">
<chunk>
<reader="cvsFileItemReader">
<writer="xmlItemWriter">
<processor="itemProcessor" commit-interval="10">
</chunk>
</step>
</job>
The classes we support in JSR-352 framework are:
method | interface |
---|---|
readItem() |
javax.batch.api.chunk.AbstractItemReader |
processItem() | javax.batch.api.chunk.ItemProcessor |
writeItem() | javax.batch.api.chunk.AbstractItemWriter |
process(), stop() | javax.batch.api.AbstractBatchlet javax.batch.api.Batchlet |
beforeJob(), afterJob() | javax.batch.api.listener.JobListener |
beforeStep(), afterStep() | javax.batch.api.listener.StepListener |
Links between these methods and the corresponding JOBS and STEPS are created in the extension for JSR-352 framework.
Features of Spring Batch Framework in Java
Supported APIs:
Job APIs | Step APIs |
---|---|
org.springframework.batch.core.job.builder.SimpleJobBuilder.build | org.springframework.batch.core.step.builder.SimpleStepBuilder.build |
org.springframework.batch.core.job.builder.FlowJobBuilder.build | org.springframework.batch.core.step.builder.FlowStepBuilder.build |
org.springframework.batch.core.job.builder.JobFlowBuilder.build | org.springframework.batch.core.step.builder.JobStepBuilder.build |
org.springframework.batch.core.step.builder.AbstractTaskletStepBuilder.build | |
org.springframework.batch.core.step.builder.PartitionStepBuilder.build | |
org.springframework.batch.core.job.builder.JobBuilder.flow | org.springframework.batch.core.step.builder.StepBuilder.tasklet |
org.springframework.batch.core.step.builder.StepBuilder.partitioner | |
org.springframework.batch.core.job.builder.JobBuilder.start | org.springframework.batch.core.step.builder.SimpleStepBuilder.reader |
org.springframework.batch.core.job.builder.JobBuilderHelper.listener | org.springframework.batch.core.step.builder.SimpleStepBuilder.processor |
org.springframework.batch.core.step.builder.SimpleStepBuilder.writer | |
org.springframework.batch.core.job.builder.FlowBuilder.next | |
org.springframework.batch.core.job.builder.SimpleJobBuilder.next | |
org.springframework.batch.core.job.builder.FlowBuilder.TransitionBuilder.to | |
org.springframework.batch.core.configuration.annotation.JobBuilderFactory.get | org.springframework.batch.core.configuration.annotation.StepBuilderFactory.get |
Configuration of a Job in Java
BatchConfiguration.java
Job job = jobBuilderFactory
.get("financialsJob")
.incrementer(new RunIdIncrementer())
.flow(statementsSteps)
.end()
.build();
Configuration of a Step in Java
BatchConfiguration.java
@Bean
public Step statementsSteps(ItemReader<String> statementsReader,
ItemWriter<Map<Integer, List<StatementHeader>>> changeAlertsWriter,
ItemProcessor<? super String, ? extends Map<Integer, List<StatementHeader>>> statementsProcessor) {
return stepBuilderFactory
.get("statementsSteps")
.chunk(1)
.reader(statementsReader)
.processor(statementsProcessor)
.writer(changeAlertsWriter).build();
Define step sequence in Java
SpringBatchConfiguration.java
...
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
...
@Configuration
@EnableBatchProcessing
public class SpringBatchConfiguration {
private static Logger LOGGER = LoggerFactory.getLogger(SpringBatchConfiguration.class);
private static final String[] TOKENS = { "bookname", "bookauthor", "bookformat", "isbn", "publishyear" };
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
...
@Bean
public Step step1(ItemReader<BookRecord> csvItemReader, ItemWriter<Book> jsonItemWriter) throws IOException {
return stepBuilderFactory
.get("step1")
.<BookRecord, Book> chunk(3)
.reader(csvItemReader)
.processor(bookItemProcessor())
.writer(jsonItemWriter)
.build();
}
@Bean
public Step step2(ItemReader<BookRecord> csvItemReader, ItemWriter<BookDetails> listItemWriter) {
return stepBuilderFactory
.get("step2")
.<BookRecord, BookDetails> chunk(3)
.reader(csvItemReader)
.processor(bookDetailsItemProcessor())
.writer(listItemWriter)
.build();
}
@Bean(name = "transformBooksRecords")
public Job transformBookRecords(Step step1, Step step2) throws IOException {
return jobBuilderFactory
.get("transformBooksRecords")
.flow(step1)
.next(step2)
.end()
.build();
}
}
Listeners with Spring Batch in Java
SpringBatchConfig.java
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import javax.annotation.Resource;
import org.springframework.beans.factory.annotation.Qualifier;
import com.job.listener.methods.JobTRExecutionListener;
public class SpringBatchConfig {
@Autowired
private JobBuilderFactory jobs;
@Autowired
private StepBuilderFactory steps;
@Value("input/record.csv")
private Resource inputCsv;
@Value("file:xml/output.xml")
private Resource outputXml;
private JobTRExecutionListener listener;
...
@Bean
protected Step step1(ItemReader<Transaction> reader,
ItemProcessor<Transaction, Transaction> processor,
ItemWriter<Transaction> writer) {
return steps.get("step1").chunk(10)
.reader(reader).processor(processor).writer(writer).build();
}
@Bean(name = "firstBatchJob")
public Job job(@Qualifier("step1") Step step1) {
return jobs.get("firstBatchJob").listener(listener).start(step1).build();
}
}
Tasklets with Spring Batch in Java
SpringBatchConfigFlow .java
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import javax.annotation.Resource;
import org.springframework.beans.factory.annotation.Qualifier;
import com.jobs.scheduler.tasklets.AcquisitionTasklet;
public class SpringBatchConfigFlow {
@Autowired
private JobBuilderFactory jobs;
@Autowired
private StepBuilderFactory steps;
private AcquisitionTasklet tasklet;
@Bean
protected Step step2(ItemReader<Transaction> reader,
ItemProcessor<Transaction, Transaction> processor,
ItemWriter<Transaction> writer) {
return steps.get("step2").tasklet(tasklet).build();
}
...
}
AcquisitionTasklet .java
package com.jobs.scheduler.tasklets;
import org.springframework.batch.core.ExitStatus;
import org.springframework.batch.core.StepContribution;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.http.HttpStatus;
import com.sncf.eva.tr.jobs.rest.client.JobsClientRest;
public class AcquisitionTasklet implements Tasklet {
/
* Logger
*/
public static final org.slf4j.Logger LOG = org.slf4j.LoggerFactory.getLogger(AcquisitionTasklet.class);
public static final String PARAM_TASKLET_DATE = "date";
private JobsClientRest jobsClientRest;
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
LOG.info("Execution de la tasklet acquisition pour les paramètres : Context - {}",
getParam(chunkContext, PARAM_TASKLET_DATE));
contribution.incrementFilterCount(1);
executeServices(contribution);
return RepeatStatus.FINISHED;
}
...
Partitioner with Spring Batch in Java
The partitioner allows parallel execution of steps.
SpringbatchPartitionConfig .java
import com.baeldung.batch.model.Transaction;
import com.baeldung.batch.service.RecordFieldSetMapper;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
...
@Configuration
@EnableBatchProcessing
public class SpringbatchPartitionConfig {
@Autowired
private JobBuilderFactory jobs;
@Autowired
private StepBuilderFactory steps;
@Bean(name = "partitionerJob")
public Job partitionerJob() throws UnexpectedInputException, MalformedURLException, ParseException {
return jobs.get("partitionerJob")
.start(partitionStep())
.build();
}
@Bean
public Step partitionStep() throws UnexpectedInputException, MalformedURLException, ParseException {
return steps.get("partitionStep")
.partitioner("slaveStep", partitioner())
.step(slaveStep())
.taskExecutor(taskExecutor())
.build();
}
@Bean
public Step slaveStep() throws UnexpectedInputException, MalformedURLException, ParseException {
return steps.get("slaveStep")
.<Transaction, Transaction>chunk(1)
.reader(itemReader(null))
.writer(itemWriter(marshaller(), null))
.build();
}
...
}
Access to databases in xml and Java
Item beans provide access to databases using cursor-based and paging-based implementations in Java and XML.
Supported APIs in Java
Java API | Query Object created |
---|---|
org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider.setSelectClause | CAST_Batch_SQL_Query |
org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider.setFromClause | |
org.springframework.batch.item.database.support.AbstractSqlPagingQueryProvider.setWhereClause | |
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.setSelectClause | CAST_Batch_SQL_Query |
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.setFromClause | |
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.setWhereClause | |
org.springframework.batch.item.database.builder.JdbcPagingItemReaderBuilder.build | CAST_Batch_JDBC_Query |
org.springframework.batch.item.database.builder.JdbcPagingItemReaderBuilder.selectClause | |
org.springframework.batch.item.database.builder.JdbcPagingItemReaderBuilder.fromClause | |
org.springframework.batch.item.database.builder.JdbcPagingItemReaderBuilder.whereClause | |
org.springframework.batch.item.database.JdbcBatchItemWriter.setSql | CAST_Batch_JDBC_Query |
org.springframework.batch.item.database.builder.JdbcBatchItemWriterBuilder.sql | |
org.springframework.batch.item.database.JdbcCursorItemReader.setSql | CAST_Batch_JDBC_Query |
org.springframework.batch.item.database.builder.JdbcCursorItemReaderBuilder.sql | |
org.springframework.batch.item.database.JpaCursorItemReader.setQueryString | CAST_Batch_JPA_SQLQuery |
org.springframework.batch.item.database.HibernateCursorItemReader.setQueryString | |
org.springframework.batch.item.database.JpaPagingItemReader.setQueryString | |
org.springframework.batch.item.database.HibernatePagingItemReader.setQueryString | |
org.springframework.batch.item.database.builder.JpaPagingItemReaderBuilder.queryString | |
org.springframework.batch.item.database.StoredProcedureItemReader.setProcedureName | CAST_Batch_JDBC_Query |
org.springframework.batch.item.database.orm.JpaNativeQueryProvider.setSqlQuery | CAST_Batch_SQL_Query |
org.springframework.batch.item.database.orm.HibernateNativeQueryProvider.setSqlQuery |
Supported APIs in XML
XML TAG | Query Object created |
---|---|
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.selectClause | CAST_Batch_SQL_Query |
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.fromClause | |
org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean.whereClause | |
org.springframework.batch.item.database.JdbcCursorItemReader sql | CAST_Batch_JDBC_Query |
org.springframework.batch.item.database.JdbcBatchItemWriter sql | |
org.springframework.batch.item.database.JpaPagingItemReader queryString | CAST_Batch_JPA_SQLQuery |
org.springframework.batch.item.database.HibernateCursorItemReader queryString | |
org.springframework.batch.item.database.StoredProcedureItemReader procedureName | CAST_Batch_JDBC_Query |
Examples
Support of org.springframework.batch.item.database.JpaPagingItemReader.queryString
import org.springframework.batch.item.database.JpaPagingItemReader;
import org.springframework.batch.item.database.builder.JpaPagingItemReaderBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import javax.persistence.EntityManagerFactory;
@Configuration
public class EnfantReaderConfig {
@Value("${batch.remplacement-reco.reader-page-size:1000}")
private int READER_PAGE_SIZE;
@Bean
public JpaPagingItemReader<Enfant> enfantReader(EntityManagerFactory entityManagerFactory) {
return new JpaPagingItemReaderBuilder<Enfant>()
.name("enfantReader")
.entityManagerFactory(entityManagerFactory)
.queryString("SELECT enfant FROM Enfant enfant")
.pageSize(READER_PAGE_SIZE)
.build();
}
}
Support of org.springframework.batch.item.database.StoredProcedureItemReader.setProcedureName
package com.example.batch.config;
import javax.sql.DataSource;
import org.springframework.batch.item.database.StoredProcedureItemReader;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class BatchConfig {
@Bean
public StoredProcedureItemReader<CustomerCredit> reader(DataSource dataSource) {
StoredProcedureItemReader<CustomerCredit> reader = new StoredProcedureItemReader<>();
reader.setDataSource(dataSource);
reader.setProcedureName("sp_customer_credit");
reader.setRowMapper(new CustomerCreditRowMapper());
return reader;
}
}
Support of org.springframework.batch.item.database.StoredProcedureItemReader in xml for procedureName property tag
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:batch="http://www.springframework.org/schema/batch"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/batch
http://www.springframework.org/schema/batch/spring-batch.xsd">
<batch:job id="customerCreditJob">
<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="reader" writer="writer" commit-interval="10"/>
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="writer" class="org.springframework.batch.item.support.ListItemWriter"/>
</beans>
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="reader" class="org.springframework.batch.item.database.StoredProcedureItemReader">
<property name="dataSource" ref="dataSource"/>
<property name="procedureName" value="sp_customer_credit"/>
<property name="rowMapper">
<bean class="com.example.batch.config.CustomerCreditRowMapper"/>
</property>
</bean>
</beans>
Support of org.springframework.batch.item.database.JpaPagingItemReader in xml for queryString property tag
<batch:job id="records" incrementer="jobIncrementer">
<!--Step de création des Fiches de raccordement -->
<batch:step id="stepRecords" next="stepRecordsUpdate">
<tasklet>
<chunk
reader="recordsCreateReader"
processor="recordsProcessor"
writer="recordsCreateWriter"
commit-interval="2" />
<batch:fail on="FAILED" />
</tasklet>
</batch:step>
</batch:job>
<beans:bean id="recordsCreateReader" class="org.springframework.batch.item.database.JpaPagingItemReader">
<beans:property name="entityManagerFactory" ref="entityManagerFactory" />
<beans:property name="queryString" value="SELECT DataRecord struct FROM FileRecords file INNER JOIN file.structureActive struct WHERE file.currentState = fileRecordState.SENT" />
</beans:bean>
Support of org.springframework.batch.item.database.builder.JdbcPagingItemReaderBuilder setSelect/setFrom/setWhereClause
public SqlPagingQueryProviderFactoryBean queryProvider() {
SqlPagingQueryProviderFactoryBean provider = new SqlPagingQueryProviderFactoryBean();
provider.setSelectClause("select id, name, credit");
provider.setFromClause("from customer_credit");
provider.setWhereClause("where status=:status");
provider.setSortKey("id");
return provider;
}
Support of org.springframework.batch.item.database.JdbcCursorItemReader in xml for sql property tag
<import resource="../config/context.xml" />
<import resource="../config/database.xml" />
<batch:job id="helloWorldJob"> //helloWorldJob Job consisting of steps,steps having chunk reader, writer, processor
<batch:step id="stepInformation">
<batch:tasklet>
<batch:chunk reader="infoItemReader" writer="infoItemWriter" processor="itemProcessor" commit-interval="10"> //reader writer and processor dependency
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
...
<bean id="infoItemReader" class="org.springframework.batch.item.database.JdbcCursorItemReader" scope="step">
<property name="sql"
value="SELECT * FROM INFORMATIONS"/>
</bean>
Compatibility
Core release | Operating System | Supported |
---|---|---|
v3/8.4.x | Microsoft Windows / Linux | ✅ |
v2/8.3.x | Microsoft Windows | ✅ |
Dependencies with other extensions
Some CAST extensions require the presence of other CAST extensions in order to function correctly. The Spring Batch extension requires that the following other CAST extensions are also installed:
- com.castsoftware.java.hibernate (≥ 1.0.7)
- com.castsoftware.internal.platform (internal technical extension)
What results can you expect?
Once the analysis/snapshot generation has completed, you can visualize the results in the usual manner:
- Spring Batch framework
- JSR-352 framework
Objects
Icon | Object Type |
---|---|
Batch Job | |
Batch Step | |
Batch SQL Query | |
Batch JDBC Query | |
Batch JPQL Query |
In order to avoid duplication and confusion, when using the Spring Batch extension, any Spring Batch Job and Spring Batch Step objects created by the JEE analyzer will be deleted from the Analysis schema at application level.
Links
The following call links are added:
- Between Batch Jobs and Batch Steps
- Between Batch Jobs and beforeJob() and afterJob() methods defined in JobExecutionListener classes
- Between the methods calling jobs and Batch Jobs
- Between Batch Steps and other Batch Steps
- Between Batch Steps and execute() methods defined in Tasklet classes
- Between Batch Steps and process() methods defined in ItemProcessor, ItemScriptItemProcessor classes
- Between Batch Steps and read(), open(), update(), close() methods defined in ItemReader and ItemStreamReader classes
- Between Batch Steps and write(), open(), update(), close() methods defined in ItemWriter and ItemStreamWriter classes
- Between Batch Steps and beforeStep(), afterStep(), beforeWrite(), afterWrite(), onWriteError(), beforeProcess(), afterProcess(), onProcessError(), beforeRead(), afterRead(), onReadError(), beforeChunk(), afterChunk(), afterChunkError() methods defined in StepExecutionListener, ItemWriteListener, ItemProcessListener, ItemReadListener, ChunkListener classes
- Between Batch Steps and Spring Beans if beans have sql properties and Spring Beans are linked to tables
Important information
- When using the Spring Batch extension, any Spring Batch Job and Spring Batch Step objects created by the JEE analyzer will be deleted from the Analysis schema at application level.
- Support of Java Beans containing SQL queries in Spring Batch framework : links between Batch Step and Spring Bean containing queries are added if Spring Bean are linked to tables.