Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
None
-
None
Description
MergeContent exposes a "Maximum number of Bins" property. However, the way in which this property is used limits only the maximum number of 'in progress' bins. Once a bin has been declared 'ready' (i.e., the FlowFiles have been grouped together and are just waiting to be written out), that 'ready bin' is not counted toward the maximum number of allowable bins.
As a result, if we have a very high volume of FlowFiles coming in, we could end up with MergeContent creating many bins and buffering potentially millions of FlowFiles, exhausting Java Heap and preventing backpressure and age-off from kicking in.