Upon further digging, it looks like SWF stands for 'Amazon Simple Workflow Service.' It basically starts, runs, and retains workflow executions, as well as schedules tasks, adds markers, receives signals, and starts timers for those workflow executions.
To overcome these issues and to support on-line data processing, even for those workflow engines that do not support pipelining, we suggest the following strategy.
The most complicated thing of the entire service is the back end, call it the engine room, the plumbing, making all of the proprietary systems that we have in place talk to each other, work with each other and get those workflows to actually work.
Those workflows receive a higher weight, leading to higher support in the recommendations of their paths.
More importantly, companies will have to have unique workflows, be at a scale where optimizing those workflows is imperative, and realize that they can match those workflows to specific hardware to maximize their cost performance.
Then those workflows are passed off to their workforce, and users can pick up whatever tasks they want that are available.
In addition, in failing to support those workflows, Adobe has made room for competition to find their own foothold in a space that should have been Adobe's to lose.
"These applications carry data-intensive workloads that demand HPC-like resources, and as more companies leverage data to support their businesses, the need to properly containerize and support those workflows has grown substantially," Kurtzer wrote in a blog post announcing the enterprise product.
As bioinformatic workflows become increasingly complex and involve multiple specialized tools, so does the difficulty of reliably reproducing those workflows.
The aim for each customised instance is to provide only those interfaces, workflow engines, workflow and workflow engine repositories, web services, discovery services, monitoring services and DCI services required by the workflows that end users plan to run.
The end users of CDS are those whose workflow is most affected by it.
