Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-5997

If swap file written but FlowFile Repository fails to update, connection queue counts wrong and flowfiles are duplicated upon restart

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.8.0, 1.7.1
    • Fix Version/s: 1.9.0
    • Component/s: None
    • Labels:
      None

      Description

      If a queue writes out a Swap File but then the FlowFile Repository throws an Exception when attempting to update, we end up with a scenario where the size of the queue increases by 10,000 FlowFiles (the number of FlowFiles to be written to the swap file) as well as the corresponding size of the FlowFiles. We also have a Swap File that is written out to disk but the FlowFile Repo didn't get updated so on restart we have those FlowFiles in the FlowFile Repo as well as in the Swap File, so we end up with two of the same FlowFile. This can then cause some odd behavior because two FlowFiles exist with the same ID and the counts on the queues are very wrong, which also causes a lot of confusion.

        Attachments

          Activity

            People

            • Assignee:
              markap14 Mark Payne
              Reporter:
              markap14 Mark Payne
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h
                1h