Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
For some flows, it is imperative that the flow files are processed in a certain order. The PriorityAttributePrioritizer can be used on a connection to ensure that flow files going through that connection are in priority order, but depending on error-handling, branching, and other flow designs, it is possible for flow files to get out-of-order.
I propose an EnforceOrder processor, which would be single-threaded and have (at a minimum) the following properties:
1) Order Attribute: This would be the name of a flow file attribute from which the current value will be retrieved.
2) Initial Value: This property specifies an initial value for the order. The processor is stateful, however, so this property is only used when there is no entry in the state map for current value.
The processor would store the Initial Value into the state map (if no state map entry exists), then for each incoming flow file, it checks the value in the Order Attribute against the current value. If the attribute value matches the current value, the flow file is transferred to the "success" relationship, and the current value is incremented in the state map. If the attribute value does not match the current value, the session will be rolled back.
Using this processor, along with a PriorityAttributePrioritizer on the incoming connection, will allow for out-of-order flow files to have a sort of "barrier", thereby guaranteeing that flow files transferred to the "success" relationship are in the specified order.
Attachments
Issue Links
- links to
