[NIFI-1577] NiFi holds open too many files when using a Run Duration > 0 ms and calling session.append

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Component/s: Core Framework
• Assignee: Mark Payne
• Reporter: Mark Payne
• Created: 29/Feb/16 20:24
• Updated: 01/Mar/16 16:53
• Resolved: 01/Mar/16 16:53

Description

If a Processor calls ProcessSession.append() and has a Run Duration scheduled > 0 ms, we quickly end up with "Too many open files" exceptions.

This appears to be due to the fact that calling append() holds the content repository's stream open so that the session can keep appending to it, but on checkpoint() the session does not close these streams. It should close these streams on checkpoint, since the Processor is no longer allowed to reference these FlowFiles anyway at that point.