Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
When all FlowFiles in a given queue are penalized, the destination processor is still scheduled to run. As a result, the Processor is triggered, no data is available, and it generally returns. However, this means that the UI shows millions of tasks completed over a short period of time, when no work was actually done.
Further, it means that we constantly schedule the processor to run as fast as it can, which wastes compute resources. Instead, we should detect that the head of the queue is penalized (and therefore all FlowFiles in the queue are penalized), and treat it the same as we would if the queue were empty, which allows the "bored yield duration" to take effect, significantly reducing CPU cycles.
