High performance computations in workflow

Learn how high performance computations are achieved in a workflow

Last published at: April 18th, 2024

Whether it's report generation, data collection for reports, or hierarchical computations these are long-running computations that take time and many computing resources to gather that data. Some of these can be done on the database for performance, but some might have to happen at the business logic level. 

Normally business logic holds the computational logic and depending on the algorithm implemented performance can vary.  Just to give you an example of this, there are many ways to compute all possible paths of a workflow.  You can use something like the "Walk the tree" algorithm, where you start from the starting node and keep walking down a path until no more nodes are available.  This method works 100%, but performance-wise it was not the greatest.  Then there are matrix-based algorithms, where you identify nodes and vertices into an N x N matrix and perform computations on the matrix to figure out all the possible paths.  This is one of the fastest algorithms for finding all paths of a graph, almost 1000x times faster than alternate options. 

FlowWright lets you expand the product by writing custom steps, custom steps are computational code that has simple to complex logic that's able to process business logic.  A custom step also utilizes multithreading concepts of parallel execution using multiple CPU cores.  Custom step's architecture is designed for performance, performance is key.

High-performance computations can also be utilized within ESB event handlers.  Event handlers are the glue between systems for processing messages placed on FlowWright's Enterprise service bus. Since the ESB events are processed by a separate ESB engine that has its robotic workers, implementing complex logic and computations will reduce the impact on the workflow engines. Also, the ESB is a nice way for the workflow to offload processing, at multiple places of the workflow, events can be generated to the ESB, and these events will be processed by ESB robotic workers in parallel.

Many times we see customers have complex and long computations and business logic built into their application APIs; you can easily access that functionality by creating a business object in FlowWright.  Business objects are simply wrappers around your code, but these business objects can be used from anywhere within a workflow process.  If you use business objects correctly, you only need to maintain business logic in 1 place.  In certain instances, some customers also call their APIs from custom steps to perform the computations. 

Data collection for computations can be a tedious process, especially when the data comes from many systems, based on how complex the algorithm is.  Complex algorithms can take up many CPU resources, but in data collection and computing processes that also depend on the network to access data from another system, a faster network can also help in this case.  Some of the report computations that FlowWright performs for projections take a lot of computing resources, but to generate faster reports, FlowWright caches data and summary data in database tables.  This improves most of the report generation speeds, storage is very cheap today, but network performance still can be a bottleneck if not optimized for speed.

As with any other piece of software, performance matters, but also performance within steps, computations, and workflows matters in your day-to-day user experience.  When executing 1000s of workflow instances at one time, performance matters.  One of our customers in Chicago, before implementing FlowWright, their report generation took 6 hrs to prepare, after implementing FlowWright into their report generation processes, now it takes 1 hr.  They have gone from 4 times a day to 24 times a day to processing reports for their customers, which improves our user's experience for the better.