Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-42101

Wrap InMemoryTableScanExec with QueryStage

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.5.0
    • 3.5.0
    • SQL
    • None

    Description

      The first access to the cached plan which is enable AQE is tricky. Currently, we can not preserve its output partitioning and ordering.

      The whole query plan also missed lots of optimization in AQE framework. Wrap 
      InMemoryTableScanExec  to query stage can resolve all these issues.

      Attachments

        Activity

          People

            ulysses XiDuo You
            ulysses XiDuo You
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: