Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6215

Cannot swap in Flowfiles

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.9.1, 1.9.2
    • Fix Version/s: 1.10.0
    • Component/s: Core Framework
    • Labels:
      None
    • Environment:
      Windows Server 2012 R2 Standard

      java version "1.8.0_181"
      Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
      Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

      Description

      Upon creating a lot of FlowFiles simultaneously (though SplitFile or ListSFTP) into a funnel, Nifi reports:

      12:15:24 CEST WARNING
      Cannot swap in FlowFiles from location C:\PROGRA~1\APACHE~1\NIFI-1~1.2\..\repositories\flowfile_repository\swap\1555323318624-9c6fa68f-38fd-3e05-6f82-1aa48b9995fb-f481b847-ff99-47a9-ab06-6501fb1e9f93.swap because the FlowFile Repository does not know about this Swap Location. This file should be manually removed. This typically occurs when a Swap File is written but the FlowFile Repository is not updated yet to reflect this. This is generally not a cause for concern, but may be indicative of a failure to update the FlowFile Repository.

      12:15:30 CEST WARNING
      Cannot swap in FlowFiles from location C:\PROGRA~1\APACHE~1\NIFI-1~1.2\..\repositories\flowfile_repository\swap\1555323319139-9c6fa68f-38fd-3e05-6f82-1aa48b9995fb-e7ec580a-dca0-4dec-a65e-9efb468f704d.swap because the FlowFile Repository does not know about this Swap Location. This file should be manually removed. This typically occurs when a Swap File is written but the FlowFile Repository is not updated yet to reflect this. This is generally not a cause for concern, but may be indicative of a failure to update the FlowFile Repository.

      [...]

      etc.

      After handling the first 20,000 files the rest seem to get 'stuck' in the swapfiles and do not get processed. Restarting Nifi erases all files in the queue and clearing the queue in some other fashion is not possible.

      Using a funnel seems to trigger the above message, without the funnel the same behaviour occurs but without the message (the swapfiles do get created).

      It was not possible to reproduce this behaviour in Unix, so it might be windows-specific, possibly though some weird race-condition.

      Attached are a screenshot of what the erroneous state looks like, as well as a template and input file that were able to reproduce the issue.

        Attachments

        1. SwapBug.log
          12 kB
          Bluemine
        2. testfile.csv
          3.15 MB
          Bluemine
        3. Capture.PNG
          315 kB
          Bluemine
        4. SwapBug.xml
          26 kB
          Bluemine

          Issue Links

            Activity

              People

              • Assignee:
                markap14 Mark Payne
                Reporter:
                Bluemine Bluemine
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h