Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-3582

Make RelStructuredTypeFlattener configurable in PlannerImpl

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      There is a use case where users want to mix Beam programming model with Beam SQL together to process a dataset. The following is an example of the use case:

      dataset.apply(something user defined)
      .apply(SELECT row_a, row_b ...)
      .apply(something user defined)

      As you can see, after the SQL statement is applied, the data structure should be preserved for further processing(e.g. users expect to see there are two rows, than flattened fields of rows)

      So far as RelStructuredTypeFlattener is enabled by default in PlannerImpl, Row/Struct will be flattened. But users are very likely defined an output scheme that include row/struct, as that's what they processed in SQL statement.

      Make calling RelStructuredTypeFlattener configurable in PlannerImpl to allow disabling it when not needed can be a solution.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                amaliujia Rui Wang
                Reporter:
                amaliujia Rui Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: