[NIFI-7843] Recursive avro schemas fail to write with RecordWriter

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.13.0
• Assignee: Denes Arvay
• Reporter: Denes Arvay
• Created: 24/Sep/20 05:14
• Updated: 28/Sep/20 20:28
• Resolved: 28/Sep/20 20:28

Description

Recursive Avro record with a schema containing self-reference cannot be written by RecordWriter due to infinite loop / StackOverflowError.

Schema:

{
  "namespace": "dataformat.avro.recursive",  
  "type": "record",  
  "name": "PersonRecursive",  
  "fields": [
    {"name": "id", "type": "int"},
    {"name": "name",  "type": ["string", "null"]},
    {"name": "parent",  "type": ["PersonRecursive", "null"]}  
  ]
} 

Exception:

2020-09-24 06:13:21,978 ERROR [Timer-Driven Process Thread-7] o.a.n.processors.standard.ConvertRecord ConvertRecord[id=b666a7ea-0174-1000-894b-7b743c091dc0] ConvertRecord[id=b666a7ea-0174-1000-894b-7b743c091dc0] failed to process session due to java.lang.StackOverflowError; Processor Administratively Yielded for 1 sec: java.lang.StackOverflowError
java.lang.StackOverflowError: null
        at java.base/java.util.Objects.equals(Objects.java:77)
        at org.apache.nifi.serialization.record.type.RecordDataType.equals(RecordDataType.java:61)
        at org.apache.nifi.serialization.record.RecordField.equals(RecordField.java:130)
        at java.base/java.util.Objects.equals(Objects.java:77)
        at java.base/java.util.ArrayList.equalsRange(ArrayList.java:574)
        at java.base/java.util.ArrayList.equals(ArrayList.java:561)
        at java.base/java.util.Collections$UnmodifiableList.equals(Collections.java:1307)
        at org.apache.nifi.serialization.SimpleRecordSchema.equals(SimpleRecordSchema.java:175)
        at java.base/java.util.Objects.equals(Objects.java:77)
        at org.apache.nifi.serialization.record.type.RecordDataType.equals(RecordDataType.java:61)
        at org.apache.nifi.serialization.record.RecordField.equals(RecordField.java:130)
        at java.base/java.util.Objects.equals(Objects.java:77)
        at java.base/java.util.ArrayList.equalsRange(ArrayList.java:574)
        at java.base/java.util.ArrayList.equals(ArrayList.java:561)
        at java.base/java.util.Collections$UnmodifiableList.equals(Collections.java:1307)
        at org.apache.nifi.serialization.SimpleRecordSchema.equals(SimpleRecordSchema.java:175)
        at java.base/java.util.Objects.equals(Objects.java:77)
        at org.apache.nifi.serialization.record.type.RecordDataType.equals(RecordDataType.java:61)
        at org.apache.nifi.serialization.record.RecordField.equals(RecordField.java:130)
        at java.base/java.util.Objects.equals(Objects.java:77)
        at java.base/java.util.ArrayList.equalsRange(ArrayList.java:574)
        at java.base/java.util.ArrayList.equals(ArrayList.java:561)
        at java.base/java.util.Collections$UnmodifiableList.equals(Collections.java:1307)
        at org.apache.nifi.serialization.SimpleRecordSchema.equals(SimpleRecordSchema.java:175)
        at java.base/java.util.Objects.equals(Objects.java:77)
        at org.apache.nifi.serialization.record.type.RecordDataType.equals(RecordDataType.java:61)
        at org.apache.nifi.serialization.record.RecordField.equals(RecordField.java:130)
        at java.base/java.util.Objects.equals(Objects.java:77)
        at java.base/java.util.ArrayList.equalsRange(ArrayList.java:574)
        at java.base/java.util.ArrayList.equals(ArrayList.java:561) 

Issue Links

• links to: GitHub Pull Request #4550