Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.0.0, 0.7.0
-
None
Description
Consider the scenario where multiple queues lead to a single component and all of them are full due to back pressure. With the attached template, it is easily observable that once a single queue starts to drain due to relieved back pressure, it will continue to drain as long as it has incoming flow files. This means that if there's a constant flow of incoming flow files to this queue, the other queues will never be drained (at least, that's my theory based on several hours of observation).
To reproduce this:
- Load the template into NiFi 1.0.0
- Play all three GenerateFlowFile processors, but not the UpdateAttribute processor (this simulates backpressure). Wait until each queue has 1,000 flow files (max backpressure)
- Stop the GenerateFlowFile processors, and play the UpdateAttribute processor (this relieves the backpressure)
- Observe which queue has started to drain, and start its GenerateFlowFile processor
- Observe that the other two queues remain full indefinitely, while the draining queue continues to replenish and be drained indefinitely
Attachments
Attachments
Issue Links
- is related to
-
NIFI-2751 When a processor pulls a batch of FlowFiles, it keeps pulling from the same inbound connection instead of round-robin'ing
- Resolved