Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-9056

Content Repository Filling Up

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.13.2
    • 1.15.0
    • Core Framework
    • None

    Description

      We have a clustered nifi setup that has recently been upgraded to 1.13.2 from 1.11.4. After upgrading, one of the issues we have run into is that the Content Repository will fill up to the nifi.content.repository.archive.backpressure.percentage mark and lock the processing & canvas. The only solution is to restart nifi at this point. We have the following properties set:

      nifi.content.repository.archive.backpressure.percentage=95%

      nifi.content.repository.archive.max.usage.percentage=25%

      nifi.content.repository.archive.max.retention.period=2 hours

      The max usage property seems to be completed ignored. Monitoring the nifi cluster disk % for content repository shows that it slowly fills up over time and never decreasing. If we pause the input to entire nifi flow and let all the processing clear out with 0 flowfiles remaining on the canvas for 15+ minutes, the content repository disk usage does not decrease. Currently, our only solution is to restart nifi on a daily cron schedule. After restarting the nifi, it will clear out the 80+ GB of the content repository and usage falls down to 0%. 

       

      There seems to be an issue removing the older content claims in 1.13.2.

      Thanks!

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              aheys Andrew Heys
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: