[NIFI-11786] JsonTreeReader with InferSchema generating NullPointerException for Specific inputs

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.21.0
• Component/s: Core Framework
• Assignee: Unassigned
• Reporter: Sanghavi
• Created: 07/Jul/23 13:10
• Updated: 07/Jul/23 13:10

Description

As part of a Dataflow, we were trying to split the list of controller services (response from API - /nifi-api/flow/controller/controller-services) using SplitRecord processor with JsonTreeReader and JsonRecordSetWriter controller services (both running on Infer schema mode).

During which we noticed that, among the list of controller service, if a controller service record having reference components containing "bulletins" field populated then schema is generating "bulletins" : "ARRAY[null]" instead of "bulletins" : "ARRAY[RECORD]".

Due to which isCompatibleDataType function of DataTypeUtils is unable to process null datatype resulting in NullPointerException.

Please find below the stack trace:-
_{java.lang.NullPointerException: null
    at org.apache.nifi.serialization.record.util.DataTypeUtils.isCompatibleDataType(DataTypeUtils.java:282)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.isArrayTypeCompatible(DataTypeUtils.java:786)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.isCompatibleDataType(DataTypeUtils.java:284)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.isRecordTypeCompatible(DataTypeUtils.java:693)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.isCompatibleDataType(DataTypeUtils.java:307)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.isArrayTypeCompatible(DataTypeUtils.java:786)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.isCompatibleDataType(DataTypeUtils.java:284)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.isCompatibleDataType(DataTypeUtils.java:278)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.chooseDataType(DataTypeUtils.java:339)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:226)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:154)
    at org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:150)
    at org.apache.nifi.json.JsonTreeRowRecordReader.convertField(JsonTreeRowRecordReader.java:259)
    at org.apache.nifi.json.JsonTreeRowRecordReader.convertJsonNodeToRecord(JsonTreeRowRecordReader.java:168)
    at org.apache.nifi.json.JsonTreeRowRecordReader.convertField(JsonTreeRowRecordReader.java:253)
    at org.apache.nifi.json.JsonTreeRowRecordReader.convertJsonNodeToRecord(JsonTreeRowRecordReader.java:168)
    at org.apache.nifi.json.JsonTreeRowRecordReader.convertJsonNodeToRecord(JsonTreeRowRecordReader.java:114)
    at org.apache.nifi.json.JsonTreeRowRecordReader.convertJsonNodeToRecord(JsonTreeRowRecordReader.java:105)
    at org.apache.nifi.json.AbstractJsonRowRecordReader.nextRecord(AbstractJsonRowRecordReader.java:181)
    at org.apache.nifi.serialization.RecordReader.nextRecord(RecordReader.java:50)
    at org.apache.nifi.serialization.RecordReader$1.next(RecordReader.java:96)
    at org.apache.nifi.serialization.record.PushBackRecordSet.next(PushBackRecordSet.java:43)
    at org.apache.nifi.serialization.record.PushBackRecordSet.isAnotherRecord(PushBackRecordSet.java:59)
    at org.apache.nifi.processors.standard.SplitRecord$1.process(SplitRecord.java:164)
    at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2692)
    at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2660)
    at org.apache.nifi.processors.standard.SplitRecord.onTrigger(SplitRecord.java:153)
    at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
    at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1360)
    at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
    at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
    at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)}
 

This setup was tested in 1.18.0, and with same configuration we are getting expected results.