Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6790

Load-balanced queues cause huge number of open pipes

    XMLWordPrintableJSON

Details

    Description

      Even very basic flows using load balance connecting queues result in huge numbers of open pipes which don't seem to get closed and result in the entire cluster being brought to a standstill because of "too many open files".  I tried apples-to-apples with equivalent very basic flows, one load balanced and one not, to show how easy this has been to reproduce.  I'm attaching some screenshots that show the "too many open files" issue in action in my NiFi cluster, along with the output of LSOF for the primary node of a cluster with vs. without load balancing.  When load balancing is enabled there is a huge number of 'pipe' and 'eventpoll' items, without load balancing things are a bit more balanced.

      Attachments

        1. lsof_loadbalanced.png
          74 kB
          Matthew Knight
        2. lsof_notloadbalanced.png
          69 kB
          Matthew Knight
        3. flow_exception.png
          249 kB
          Matthew Knight

        Issue Links

          Activity

            People

              Unassigned Unassigned
              Matthew Knight Matthew Knight
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 672h
                  672h
                  Remaining:
                  Remaining Estimate - 672h
                  672h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified