Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-12305

Flatten SMT fails on arrays

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.0.1, 2.1.1, 2.2.2, 2.3.1, 2.4.1, 2.5.1, 2.7.0, 2.6.1, 2.8.0
    • None
    • KafkaConnect
    • None

    Description

      The Flatten SMT fails for array types. A sophisticated approach that tries to flatten arrays might be desirable in some cases, and may have been punted during the early design phase of the transform, but in the interim, it's probably not worth it to make array data and the SMT mutually exclusive.

      A naive approach that preserves arrays as-are and doesn't attempt to flatten them seems fair for now, but one alternative could be to traverse array elements and, if any are maps or structs, flatten those as well.

      Adding behavior to fully flatten arrays by essentially transforming them into maps whose elements are the elements of the array and whose keys are the indices of each element is likely out of scope for a bug fix and, although useful, might have to wait for a KIP.

      Attachments

        Issue Links

          Activity

            People

              ChrisEgerton Chris Egerton
              ChrisEgerton Chris Egerton
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: