Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23375

Optimizer should remove unneeded Sort

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.3.0
    • 2.4.0
    • SQL
    • None

    Description

      As pointed out in SPARK-23368, as of now there is no rule to remove the Sort operator on an already sorted plan, ie. if we have a query like:

      SELECT b
      FROM (
          SELECT a, b
          FROM table1
          ORDER BY a
      ) t
      ORDER BY a
      

      The sort is actually executed twice, even though it is not needed.

      Attachments

        Activity

          People

            mgaido Marco Gaido
            mgaido Marco Gaido
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: