[NIFI-6065] Root Group Ports "waiting" too long

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.9.0
• Fix Version/s: 1.10.0, 1.9.1
• Component/s: Core Framework
• Assignee: Mark Payne
• Reporter: Brandon Rhys DeVries
• Created: 21/Feb/19 17:00
• Updated: 12/Mar/19 16:27
• Resolved: 22/Feb/19 15:49

Description

In NIFI-408, a change was made to StandardRootGroupPort to poll for 100 millis instead of doing a non-blocking poll, and yielding if there were no results.  While I don't know the initial motivation for this change (thoughts, markap14?), the current implementation is resulting in an excessive number of Timer-Driven threads stuck in the TIMED_WAITING state.  Essentially, if you have a high number of low traffic remote ports, you lose a substantial chunk of your timer driven thread pool to simply "waiting".

My first thought would be to revert the change, possibly even without the context.yield.  Second would be to drastically reduce the poll timeout... 10 or 5 millis, although any choice is somewhat arbitrary and flow-dependent. EDIT: which probably makes it a candidate for a nifi.property 

Any other thoughts or suggestions would be appreciated.

Issue Links

• links to: GitHub Pull Request #3328