Uploaded image for project: 'Apache NiFi MiNiFi C++'
  1. Apache NiFi MiNiFi C++
  2. MINIFICPP-1274

Flow restart could double-spend flowfiles

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None

    Description

      Flowfiles are async-deleted from the FlowFileRepository and no flush happens after shutdown, leaving these marked files in the repository. If we restart the agent, this allows these zombie files to be resurrected and be put back into their last connections. This could cause files to be processed multiple times even if we marked them for deletion.

      Solution proposal: flush the FlowFileRepository after shutdown, so all marked files are actually deleted (this won't save us from double-processing flowfiles after a crash)

      (also make sure that the FlowFileRepository shutdown happens after no more processors are running)

      Attachments

        Activity

          People

            adebreceni Adam Debreceni
            adebreceni Adam Debreceni
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h
                3h