This entry is the first in a series of posts about how to process different steps in Spring Batch remotely. In this project we are going to use Amazon SQS as communication channel between the remote processes.
Spring Batch is a processing framework designed for robust execution of jobs. Each job is divided in steps and each step could be executed in a remote way using slaves. There are differents strategies to do this:
- Remote Chunking: This strategy is useful when we don’t have bottleneck in reading or writing. Spring Batch AWS Series (II): Remote Chunking
- Remote Partitioning: This strategy is useful when the bottleneck is in reading or writing. Spring Batch AWS Series (III): Remote Partitioning
The goal of this project is create a Job with diferent steps and each one will implement a different remote strategy.
- Step1: Using Remote Chunking
- Step2: Using Partitioning with pooling
We are going to divide in two projects:
masterproject that will be in charge of orchestrating the Job and sending the tasks to the slaves.
slaveswill do the work and will report the result to master.
 Spring Batch Framework Github