[NIFI-8620] NullPointerException on commit with multiple ProcessSession

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.13.0
• Fix Version/s: 1.14.0
• Component/s: Core Framework
• Environment: Kubernetes on linux
• Assignee: Mark Payne
• Reporter: Stanislas Deneuville
• Created: 20/May/21 11:08
• Updated: 14/Jun/21 14:01
• Resolved: 14/Jun/21 14:01

Description

For my particular use case I read and process a big file and produce smaller results along the way. I wanted to be able to regularly commit what has already been done before the end of the whole process.

So I inspired myself from the BinManager and created a custom processor that use 2 ProcessSessions:

• a first main session for the read input flowfile and creating new ones forked from it
• a second session for committing on the fly
   

The workflow is something like that:

final ProcessSession mainSession = sessionFactory.createSession(); 
final ProcessSession secondSession = sessionFactory.createSession();

FlowFile inputFlowFile = mainSession.get();

try (InputStream in = mainSession.read(inputFlowFile)) {
  while (stillSomethingToRead) {
    // read and process data
    inputData = in.read(...);
    transformedData = transform(inputData);

    // Create output flowfile
    FlowFile outputflowFile = mainSession.create(inputFlowfile);
    // write transformedData to outputflowFile content
    [...]
    // also put some attributes on outputflowFile
    [...]

    // Output the results without waiting
    mainSession.migrate(secondSession, outputflowFiles);
    secondSession.transfer(outputflowFiles, successRelationship);
    secondSession.commit();
  }
}
mainsession.commit()

 

It works well on Nifi Mock, however in a real Nifi environment I get a null pointer exception during the commit.

[id=9f6342ac-ae78-30f7-22cf-6d7517618f19] Unknown error occurred: java.lang.NullPointerException java.lang.NullPointerException: null
        at org.apache.nifi.controller.repository.StandardProcessSession.updateEventContentClaims(StandardProcessSession.java:786)
        at org.apache.nifi.controller.repository.StandardProcessSession.updateProvenanceRepo(StandardProcessSession.java:600)
        at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:353)
        at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:332)

 
Note: I don't do anything related to Data Provenance in my code.

Issue Links

• links to: GitHub Pull Request #5099