Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-8659

JoltTransformRecord should support transformation of one record to multiple output records

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.14.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently the JOLT transformation in JoltTransformRecord is applied to each record individually, and a single record is expected and written out. However there are some cases where multiple output records are desired for each input record, such as "hoisting" a nested array of records up to the top-level (without a top-level key which would make it a single record). For example:

      Input record:
      {"x": [

      {"a": 1, "b":2}

      ,

      {"a": 3, "b": 4}

      ,

      {"a": 5, "b": 6}

      ]}

      Desired output (3 records):
      [

      {"a": 1, "b":2}

      ,

      {"a": 3, "b": 4}

      ,

      {"a": 5, "b": 6}

      ]

      This would currently still require the schema of the array elements to be identical, as the first transformed record's schema will be used to write all the output records.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mattyb149 Matt Burgess
                Reporter:
                mattyb149 Matt Burgess
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m