[NIFI-579] MergeContent should handle 10s of thousands+ item merges more efficiently

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Component/s: Extensions
• Assignee: Unassigned
• Reporter: Joe Witt
• Created: 04/May/15 05:06
• Updated: 03/Jun/15 16:10
• Resolved: 03/Jun/15 16:10

Description

The way MergeContent works it opens the file handles for all items to be merged then writes all their content to a new resulting output file.

It creates problems when merging thousands, tens of thousands, and more objects as it means there are massive amounts of open file handles. This seems avoidable and users should not be required to leave unlimited number of open file handles for it to work nicely.

Issue Links

• is duplicated by: NIFI-516 MergeContent when merging 10,000s or more objects can cause too many open files