Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-3066

MergeContent: "Cannot migrate FlowFiles from a Process Session to itself"

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 1.1.0
    • 1.1.0
    • None
    • None

    Description

      With the following properties in MergeContent, I get this error when I try to process a flow file (which is 135MB):

      Merge Strategy = Bin-Packing Algorithm
      Merge Format = ZIP
      Attribute Strategy = Keep Only Common Attributes
      Correlation Attribute Name = bundle.identifier
      Minimum Number of Entries = 200
      Maximum Number of Entries = 200
      Minimum Group Size = 20 MB
      Maximum Group Size = 20 MB
      Max Bin Age = 1 min
      Maximum number of Bins = 100
      Delimiter Strategy = Filename
      Keep Path = false

      016-11-18 18:00:13,752 ERROR [Timer-Driven Process Thread-83] o.a.n.processors.standard.MergeContent
      java.lang.IllegalArgumentException: Cannot migrate FlowFiles from a Process Session to itself
              at org.apache.nifi.controller.repository.StandardProcessSession.migrate(StandardProcessSession.java:1091) ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
              at org.apache.nifi.processor.util.bin.Bin.offer(Bin.java:142) ~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
              at org.apache.nifi.processor.util.bin.BinFiles.binFlowFiles(BinFiles.java:282) ~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
              at org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:178) ~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
              at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1089) ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
      ...
      2016-11-18 18:00:13,753 WARN [Timer-Driven Process Thread-83] o.a.n.processors.standard.MergeContent MergeContent[id=a9476272-dab0-3d2f-acef-abf11f7d1b80] Processor Administratively Yielded for 1 sec due to processing failure
      

      However, other smaller flow files are able to successfully go through the processor. Not sure if it's related to the size of the flow file or not. Perhaps there just needs to be a check around this line in Bin.java:142, to make sure we don't try to migrate a flow file to its own session:

      session.migrate(getSession(), Collections.singleton(flowFile));
      

      Attachments

        Issue Links

          Activity

            People

              markap14 Mark Payne
              jgresock Joe Gresock
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: