Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-7114

DistributedMapCacheClient/Server not closing resources leading to exhausted file handles

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.10.0, 1.11.0
    • Fix Version/s: 1.12.0, 1.11.3
    • Component/s: Configuration
    • Labels:
      None
    • Environment:
      Amazon EC2 running either Amazon Linux 2 or Ubuntu 18.04.
      NiFi has been installed with no change to any configuration file.

      Description

      Since at least version 1.10 NiFi stopped closing file handles. It opens circa 500 files per hour (measured using lsof) without any apparent limit until it crashes due to too many open files.

       

      Increasing the computer open file limit is not a solution since NiFi will still crash, it'll only take longer to do so.

        Attachments

        1. destination.xml
          145 kB
          Vinicius Zolin
        2. fifocounts.txt
          1 kB
          Paul Kelly
        3. flow.xml.gz
          2 kB
          Paul Kelly
        4. lsof.log
          1.12 MB
          Vinicius Zolin
        5. lsof.zip
          7.87 MB
          Vinicius Zolin
        6. lsofAfter.log
          1.13 MB
          Vinicius Zolin
        7. lsofBefore.log
          1.12 MB
          Vinicius Zolin
        8. openFiles.xlsx
          65 kB
          Vinicius Zolin
        9. reproduction.zip
          48.04 MB
          Paul Kelly
        10. source.xml
          14 kB
          Vinicius Zolin

          Issue Links

            Activity

              People

              • Assignee:
                mattyb149 Matt Burgess
                Reporter:
                vzolin Vinicius Zolin
              • Votes:
                3 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m