[NIFI-8365] JSON record reader mishandles deep CHOICE types

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.14.0
• Assignee: Unassigned
• Reporter: Tamas Palfy
• Created: 24/Mar/21 20:14
• Updated: 19/Apr/21 16:59
• Resolved: 19/Apr/21 16:59

Description

The AbstractJsonRowRecordReader when trying to find the correct schema for a given record it may come with a wrong one.
For example:

Suppose the following record:

{
  "dataCollection":[
    {
      "record": {
        "integer": 1,
        "boolean": true
      }
    },
    {
      "record": {
        "integer": 2,
        "string": "stringValue2"
      }
    }
  ]
}

Even if the schema is correctly set (which may not be the case as infer schema itself has a similar issue),
the second record

    {
      "record": {
        "integer": 2,
        "string": "stringValue2"
      }
    }

will be assigned the schema of the first (["integer" : "INT", "boolean" : "BOOLEAN"] instead of ["integer" : "INT", "string" : "STRING"]).

This will cause the fields that are not present in the schema (in this case "string") to be omitted when writing it out.

Issue Links

• links to: GitHub Pull Request #4934