Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-8620

NullPointerException on commit with multiple ProcessSession

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.13.0
    • Fix Version/s: 1.14.0
    • Component/s: Core Framework
    • Labels:
      None
    • Environment:
      Kubernetes on linux

      Description

      For my particular use case I read and process a big file and produce smaller results along the way. I wanted to be able to regularly commit what has already been done before the end of the whole process.

      So I inspired myself from the BinManager and created a custom processor that use 2 ProcessSessions:

      • a first main session for the read input flowfile and creating new ones forked from it
      • a second session for committing on the fly
         

      The workflow is something like that:

      final ProcessSession mainSession = sessionFactory.createSession(); 
      final ProcessSession secondSession = sessionFactory.createSession();
      
      FlowFile inputFlowFile = mainSession.get();
      
      try (InputStream in = mainSession.read(inputFlowFile)) {
        while (stillSomethingToRead) {
          // read and process data
          inputData = in.read(...);
          transformedData = transform(inputData);
      
          // Create output flowfile
          FlowFile outputflowFile = mainSession.create(inputFlowfile);
          // write transformedData to outputflowFile content
          [...]
          // also put some attributes on outputflowFile
          [...]
      
          // Output the results without waiting
          mainSession.migrate(secondSession, outputflowFiles);
          secondSession.transfer(outputflowFiles, successRelationship);
          secondSession.commit();
        }
      }
      mainsession.commit()
      

       

      It works well on Nifi Mock, however in a real Nifi environment I get a null pointer exception during the commit.

      [id=9f6342ac-ae78-30f7-22cf-6d7517618f19] Unknown error occurred: java.lang.NullPointerException java.lang.NullPointerException: null
              at org.apache.nifi.controller.repository.StandardProcessSession.updateEventContentClaims(StandardProcessSession.java:786)
              at org.apache.nifi.controller.repository.StandardProcessSession.updateProvenanceRepo(StandardProcessSession.java:600)
              at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:353)
              at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:332)

       
      Note: I don't do anything related to Data Provenance in my code.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                markap14 Mark Payne
                Reporter:
                StanislasDeneuville Stanislas Deneuville
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m