Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Use Case
When merging using the defragment strategy it is frustrating when you get a spike in load and then the MergeContent processor starts routing bins to failure before they were able to defragment because the maximum number of bins are already in use and a new flowfile has arrived that doesn't fit into a current bin.
Rather than evicting the oldest bin, it would be preferable if the new flowfile that doesn't fit into an existing bin were routed to a wait relationship.
Under this feature, bins will be evicted only if they have not completed within a user-specified timeframe.
Under this feature, new flowfiles should not get stuck behind flowfiles that are waiting for a new bin to free up. If this cannot be handled in the processor code, the Usage documentation should identify requirements for prioritizers that are put on input queues.
This feature may also be helpful for the bin-packing strategy. I'll leave that up to the team to decide.