[NIFI-90] Replace explicit penalization with automatic penalization/back-off

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Core Framework
• Assignee: Unassigned
• Reporter: Joe Witt
• Created: 05/Dec/14 20:58
• Updated: 06/Mar/18 19:54

Description

Rather than having users configure explicit penalization periods and requiring developers to implement it in their processors we can automate this. Perhaps keep a LinkedHashMap<Connection ID, Counter> of size 5 or so in the FlowFileRecord construct. When a FlowFile is routed to a Connection, the counter is incremented. If the counter exceeds 3 visits to the same connection, the FlowFile will be automatically penalized. This protects us "5 hops out" so that if we have something like DistributeLoad -> PostHTTP with failure looping back to DistributeLoad, we will still penalize when appropriate.

In addition, we will remove the configuration option from the UI, setting the penalization period to some default such as 5 seconds.