[NIFI-9862] Update JsonTreeReader to read Records from a Nested Array

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.17.0, 1.16.1
• Component/s: Extensions
• Assignee: Lehel Boér
• Reporter: David Handermann
• Created: 01/Apr/22 18:09
• Updated: 19/Apr/22 16:25
• Resolved: 14/Apr/22 21:05

Description

The JsonTreeReader leverages the Jackson JSON streaming API to read one or more records from an InputStream. The supporting RecordReader implementation expects input JSON to be formatted with the an array as the root element, or an object containing the entire Record. This approach supports streamed reading of JSON objects contained within an array as the root element, but does not support streaming of JSON objects contained within an array nested inside a wrapping root object.

Some services provide JSON responses that include multiple records in a wrapping root object as follows:

{
  "total": 2,
  "records": [
    {
      "id": 1
    },
    {
      "id": 2
    }
  ]
}

In order to enable streamed processing of nested records, the JsonTreeReader should be updated to support an optional property defining the Property Name of a nested field containing records.

Issue Links

• blocks: NIFI-9720 Add processor that can retrieve records from Salesforce SObjects
• links to: GitHub Pull Request #5937