Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-122

FlowFileQueue should block if queue is too full

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.4.0
    • Core Framework
    • None

    Description

      If data comes in at extremely high rates, it's possible that the FlowFiles will not be swapped out quickly enough. This can lead to OOME. Instead, the FlowFileQueue should set a limit to the queue size and block when attempting to offer a new FlowFile until either the consuming component consumes the data or the data is swapped out via the Swap Manager (MUST ensure that we do not block while holding lock!)

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. NIFI-752 Swapping and out of FlowFiles should be driven by FlowFileQueue, rather than in background thread

          • Major - Major loss of function.
          • Resolved
          is part of

          Improvement - An improvement or enhancement to an existing feature or task. NIFI-730 Purge Queue from UI

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. NIFI-697 MergeContent and queue swapping

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              markap14 Mark Payne
              mcgilman Matt Gilman
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: