Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-5361

Upon restart of NiFi, some processors show Active Threads but never actually run

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.8.0, 1.7.1
    • Component/s: Core Framework
    • Labels:
      None

      Description

      Upon restart, if there are a large number of processors that take a long time to start up (i.e., their @OnScheduled methods take a long time to complete), then other processors may also timeout while waiting for their @OnScheduled method to complete. It looks like this is due to the fact that when we schedule many processors we determine the 'timeout timestamp' before submitting the background task to call @OnScheduled. As a result, if several processors don't finish or take a long time to finish, then other processors may just sit, waiting for their chance to have @OnScheduled called and eventually timeout before they even have a chance to invoke the method.

      When this happens, the result is that the Runnable to call @OnScheduled never is called before the corresponding Future is canceled. So the Processor is never scheduled. However, its Active Thread count has already been incremented. So it appears as if the Processor has an Active Thread but it will never make progress.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                markap14 Mark Payne
                Reporter:
                markap14 Mark Payne
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: