Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23375

Optimizer should remove unneeded Sort

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.4.0
    • Component/s: SQL
    • Labels:
      None

      Description

      As pointed out in SPARK-23368, as of now there is no rule to remove the Sort operator on an already sorted plan, ie. if we have a query like:

      SELECT b
      FROM (
          SELECT a, b
          FROM table1
          ORDER BY a
      ) t
      ORDER BY a
      

      The sort is actually executed twice, even though it is not needed.

        Attachments

          Activity

            People

            • Assignee:
              mgaido Marco Gaido
              Reporter:
              mgaido Marco Gaido
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: