[NIFI-7882] Null as default value in avro generates exception

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.17.0
• Component/s: Core Framework
• Assignee: Unassigned
• Reporter: Dominik Przybysz
• Created: 05/Oct/20 07:10
• Updated: 05/Sep/23 16:25
• Resolved: 05/Sep/23 16:25

Description

When I have a record without value for fields, then missing fields will be populated with default values from schema. It was added in https://issues.apache.org/jira/browse/NIFI-4030

But when null is default value then avro 1.8.1 returns org.apache.avro.JsonProperties$Null which cannot be converted to null value (the issue in avro is described in https://issues.apache.org/jira/browse/AVRO-1954) and processors fail with error:

ERROR o.a.n.p.standard.ConvertRecord - ConvertRecord[id=37460cbe-17f1-4456-a6b9-f4ed1baa4c45] Failed to process FlowFile[0,436030859382835.mockFlowFile,161B]; will route to failure: org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroRuntimeException: Unknown datum type org.apache.avro.JsonProperties$Null: org.apache.avro.JsonProperties$Null@1723f29f

Probably Apache Nifi should upgrade to newer avro or backport the fix from avro 1.9.x to AvroTypeUtil

Issue Links

• is related to: AVRO-1954 Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null
• is related to: NIFI-4030 Avro Schema with default values can cause NPE
• is related to: NIFI-7234 Upgrade Avro dependency from 1.8.1 to any newer version to resolve bugs in Avro
• links to: GitHub Pull Request #4574
• links to: GitHub Pull Request #4575