Saga Process Orchestration in Java Using the Flowable Process Engine
Photo by
Bootstrapping Flowable Boot App
Flowable provides a core set of open source business process engines that are compact and highly efficient. They provide a workflow and Business Process Management (BPM) platform for developers, system admins and business users. At its heart is a lightning fast, tried and tested dynamic BPMN process engine, with accompanying DMN decision tables and CMMN case management engines, all written in Java. They are Apache 2.0 licensed open source, with a committed community. (
Link
)
As one can see Flowable is positioned as a Workflow and Business Process Management Platform, what we want to understand is the feasibility of Flowable in context of low-level service orchestration, as defined as Saga Orchestration.
Refer to the following excellent article by Bernd Rücker in order to get an overview of how to tackle “
Business Transactions without two-phase commit
”
The Saga pattern describes how to solve distributed (business) transactions without two-phase-commit as this does not scale in distributed systems. The basic idea is to break the overall transaction into multiple steps or activities. Only the steps internally can be performed in atomic transactions but the overall consistency is taken care of by the Saga. The Saga has the responsibility to either get the overall business transaction completed or to leave the system in a known termination state.( Link )
In this article, I don’t address compensating transaction but will concentrate on the basic skeleton necessary to execute Saga process orchestration. It’s mainly a combination of the workflow capability provided by Flowable combined with the power of Apache Camel (a Java integration framework based on the
EIP pattern
), which is a first-citizen in Flowable via the so-called “Camel Task”.
Spring Boot Application Framework
Flowable in its nature is a Spring Boot based application and benefits from all the feature provided by Spring Boot.
Spring Boot makes it easy to create stand-alone, production-grade Spring based Applications that you can “just run”.
We take an opinionated view of the Spring platform and third-party libraries so you can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration.( Link )
Spring Boot is all about convention over configuration. Our
build.gradle
looks as follows
The gradle build file is quite compact and consists of the following dependencies:
Spring Boot framework dependencies
Flowable Spring Boot integrated framework dependencies
Camel Boot integrated framework dependencies
As well as H2 an in-memory database required for the Flowable stateful processing engine
That’s it, rather straightforward.
Spring Boot and Flowable
By introducing the above Flowable dependencies (refer to the
Flowable Spring Boot App Documentation
) and using the
@SpringBootAplication
annotation a lot has happened behind the scenes:
An in-memory datasource is created automatically (because the H2 driver is on the classpath) and passed to the Flowable process engine configuration
A Flowable ProcessEngine, CMMNEngine, DMNEngine, FormEngine, ContentEngine and IdmEngine beans have been created and exposed
All the Flowable services are exposed as Spring beans
The Spring Job Executor is created
You may fine-tune the Flowable App — i.e. in case you only need a subset of the provided engines, refer to the following
Flowable Starter List
.
SpringBoot and Camel
The Apache Camel Integration Framework is included via the following two dependencies.
implementation ‘org.apache.camel:camel-spring-boot-starter:2.23.0’
implementation ‘org.flowable:flowable-camel:6.4.1’
First dependency ties Camel into the Spring Boot App framework,
second dependency ties Camel into the Flowable Process Engine, via the so-called Camel Task, which enables the delegation of any third party system integration into a Flowable Process Model to a Camel Route (more on that later)
To get a barebone Flowable S
pring App running, follow these steps:
Introduce an application.properties with the following properties.
management.endpoints.web.exposure.include=*
management.endpoint.health.show-details=
when_authorized
flowable.idm.password-encoder=spring_delegating
Provide a SecurityConfiguration class.
“` java
The staged event-driven architecture ( SEDA) refers to an approach to software architecture that decomposes a complex, event-driven application into a set of stages connected by queues. It avoids the high overhead associated with thread-based concurrency models (i.e. locking, unlocking, and polling for locks), and decouples event and thread scheduling from application logic.
As we can see after the retrieval of the message we simulate some processing
.log(LoggingLevel. INFO, logger, "External System Processing…”)
.transform().simple(“Processed: ${property.input}, { Result: OK }”);
Finally, we return the result back to our Saga process
from(“seda:continueAsync”)
.to(“flowable:saga1Process:receiveAsyncTask”);
That’s it. So any kind of external system integration we can delegate to Camel routes, which are providing powerful integration concepts. As for example instead of implementing a retry mechanism as part of the Saga process, one can use an exception handler with a re-delivery policy also know, as the RedeliveryErrorHandler in Camel. This error handler will allow you to set the number of retries and also set things like the delay between retries etc.
public void configure() {
// ExceptionHandler with RedliveryPolicy
errorHandler(defaultErrorHandler()
.maximumRedeliveries(3)
.backOffMultiplier(4)
.retryAttemptedLogLevel(LoggingLevel. WARN));
Receive Async Response Task This Service Receiver Task will serve as the bridge back from Camel as explained above.
Save Output Task Finally, we print out the produce Camel message which was passed over.
java
package net.cloudburo.task.saga1;
import org.flowable.engine.delegate.DelegateExecution; import org.flowable.engine.delegate.JavaDelegate; import org.slf4j.Logger; import org.slf4j.LoggerFactory;
import java.util.Map;
public class SaveOutputTask implements JavaDelegate {
private Logger logger = LoggerFactory.getLogger(SaveOutputTask.class);
@SuppressWarnings("unchecked")
@Override
public void execute(DelegateExecution execution) {
Map<String, String> outputMap = (Map<String, String>) execution.getVariable("outputMap");
outputMap.put("outputValue", (String) execution.getVariable("camelBody"));
logger.info("CamelBody: " + execution.getVariable("camelBody"));
}
}
</span></span>
</div>
<div style=" font-size: 21px; text-align: start; background-color: rgb(255, 255, 255); margin-top: 1em; margin-bottom: 1em;"></div>
<span style="font-family: medium-content-sans-serif-font, "Lucida Grande", "Lucida Sans Unicode", "Lucida Sans", Geneva, Arial, sans-serif; font-weight: bold; color: rgba(0, 0, 0, 0.843137); font-size: 34px; background-color: rgb(255, 255, 255);">Spring Boot App </span>
<div style=" font-size: 21px; text-align: start; background-color: rgb(255, 255, 255); margin-top: 1em; margin-bottom: 1em;">
<span style="background-color: rgb(204, 204, 204); color: rgba(0, 0, 0, 0.843137); font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; --inversion-type-background-color: simple; --inversion-type-color: simple;"><span style=" font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; color: rgba(0, 0, 0, 0.843137); background-color: rgb(255, 255, 255);">Within the </span><span style=" font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; color: rgba(0, 0, 0, 0.843137); background-color: rgba(0, 0, 0, 0.0470588); font-style: italic;">SpringBootApplication</span><span style=" font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; color: rgba(0, 0, 0, 0.843137); background-color: rgb(255, 255, 255);"> we introduce a </span><span style=" font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; color: rgba(0, 0, 0, 0.843137); background-color: rgba(0, 0, 0, 0.0470588);">CommandLineRunner</span><span style=" font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; color: rgba(0, 0, 0, 0.843137); background-color: rgb(255, 255, 255);">which will trigger our process model. </span></span>
<br>
</div>
<div style=" font-size: 21px; text-align: start; background-color: rgb(255, 255, 255); margin-top: 1em; margin-bottom: 1em;">
<span style="background-color: rgb(204, 204, 204); color: rgba(0, 0, 0, 0.843137); font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; --inversion-type-background-color: simple; --inversion-type-color: simple;"><span style=" font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; color: rgba(0, 0, 0, 0.843137); background-color: rgb(255, 255, 255);"><code> java</span></span>
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
@SpringBootApplication
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
public class DemoApplication {
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
<br>
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
Logger logger = LoggerFactory.getLogger(DemoApplication.class);
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
<br>
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
public static void main(String[] args) {
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
SpringApplication.run(DemoApplication.class, args);
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
}
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
<br>
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
@Bean
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
public CommandLineRunner init(final RepositoryService repositoryService,
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
final RuntimeService runtimeService,
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
final TaskService taskService) {
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
<br>
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
return new CommandLineRunner() {
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
@Override
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
public void run(String... strings) throws Exception {
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
ProcessDefinitionQuery query = repositoryService.createProcessDefinitionQuery();
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
logger.info("Number of process definitions : " + query.count());
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
List<ProcessDefinition> definitions = query.list();
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
for (ProcessDefinition definition : definitions) {
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
logger.info("Loaded Process Definitions: "+definition.getKey());
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
}
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
// Execute the Saga1 Process
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
runtimeService.startProcessInstanceByKey("saga1Process");
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
}
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
};
</div>
<div style="color: rgb(0, 0, 0); text-align: start; --inversion-type-color: simple;">
}
</div>
<div style=" font-size: 21px; text-align: start; background-color: rgb(255, 255, 255); margin-top: 1em; margin-bottom: 1em;">
<span style="color: rgb(0, 0, 0); --inversion-type-color: simple;">}</span>
<br>
</div>
<div style=" font-size: 21px; text-align: start; background-color: rgb(255, 255, 255); margin-top: 1em; margin-bottom: 1em;">
<span style="background-color: rgb(204, 204, 204); color: rgba(0, 0, 0, 0.843137); font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; --inversion-type-background-color: simple; --inversion-type-color: simple;"><span style=" font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; color: rgba(0, 0, 0, 0.843137); background-color: rgb(255, 255, 255);"></code></span></span>
</div>
<div style=" font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; color: rgba(0, 0, 0, 0.843137); text-align: start; background-color: rgb(255, 255, 255); margin-top: 1em; margin-bottom: 1em;">
<span style=" font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; color: rgba(0, 0, 0, 0.843137); background-color: rgb(255, 255, 255);">f we restart our application: </span>
</div>
<div style=" font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; background-color: rgba(0, 0, 0, 0.0470588); padding: 20px; color: rgba(0, 0, 0, 0.843137); text-align: start;">
<span style=" font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; background-color: rgba(0, 0, 0, 0.0470588); color: rgba(0, 0, 0, 0.843137);">gradle bootApp</span>
</div>
<div style=" font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; color: rgba(0, 0, 0, 0.843137); text-align: start; background-color: rgb(255, 255, 255); margin-top: 1em; margin-bottom: 1em;">
<span style=" font-family: medium-content-serif-font, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; color: rgba(0, 0, 0, 0.843137); background-color: rgb(255, 255, 255);">we can see on the console the processing output of our Saga process. </span>
</div>
<div style=" text-align: start;">
<div style=" box-sizing: border-box; background-color: rgba(0, 0, 0, 0);">
<img class='article img-thumbnail' src='/images/26342dcaf983891e3fa4a4b0cbdd5fff.png'>
Last but not least you may also run unit tests via
gradle test
To wrap it up: As one can see allows Flowable with the concept of Camel Task a powerful integration of third-party systems into a BPMN process flow. The approach will ensure that the orchestration flow will not be overloaded by enterprise integration logic which is offloaded to Camel Routes.
You can check out the code here:
comments powered by Disqus