[OODT-215] Workflow2 Architecture

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.5
• Component/s: workflow manager

Environment

from JPL's internal JIRA, pre-ASF

• Assignee: Chris A. Mattmann
• Reporter: Chris A. Mattmann
• Created: 19/Jun/11 00:29
• Updated: 06/Aug/14 16:21
• Resolved: 01/Sep/12 21:14

Description

We'll use this issue to keep track of the proposal for a Workflow2 architecture, supporting more complex workflows (fan-ins and fan-outs) and more fine-grained control of the underlying control and data flow models. Here is the proposal from Brian and Paul and me:

In the new workflow model structure, a model has an execution type. cas-workflow will come standard with 4 workflow execute types implemented, but extendable so that new types can be added, current types can be extended, and even overriding of current types. The standard workflow execution types are (keep in mind that a task is considered a workflow of 1 in the new model):

1) Sequential

will run all containing workflows in sequence
2) Parallel
will run all containing workflows in parallel
3) Subset-Parallel
will run all containing workflows in parallel, but allowing for the case that some containing workflows may not run
4) Ntimes
will run all containing workflows in parallel a given number of times determined by a creation rule interface

Properties can be added to any given workflow thus allowing for a customizable workflow type extension point.

The default type will be 'Sequential' thus allowing for the old workflow structure to work seamlessly with the new version.

Example xml:

<workflow id=""> or <workflow id-ref=""> . . . the former declares that a workflow is being defined; the latter that a workflow has already been defined and is being referenced.

additional attributes include:
execution : valid values, by default, are the standard workflow execution types described above
trans-in : a list of pre-Condition ids that must be met in order for the workflow to run
trans-out : a set of post-Condition ids that must be met for successful completion of the workflow
any attribute in the namespace 'p' : this declares a properties for the workflow that can be used to customize this workflow model's execution type
<task id=""> or <task id-ref=""> . . . same as to the 'workflow' element except no 'execution' attribute is allowed — tasks are a specific type of workflow that is execution 'single' (the implied workflow execution type by the 'task' element).
also allows for all previously supported child elements allowed in old workflow model

Comments are of course, welcome.

Issue Links

• incorporates: OODT-202 NonBlockWorkflow double processing and not pausing
• incorporates: OODT-203 Create a Non-Blocking threaded implementation of the Workflow Engine
• is blocked by: OODT-311 Test failing related to workflow tasks and the DataSourceWorkflowRepository
• is related to: OODT-491 Finish line tasks for Wengine integration