Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18170

Confusing error message when using rangeBetween without specifying an "orderBy"

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Incomplete
    • None
    • None
    • SQL

    Description

      spark.range(1,3).select(sum('id) over Window.rangeBetween(0,1)).show
      

      throws runtime exception:

      Non-Zero range offsets are not supported for windows with multiple order expressions.
      

      which is confusing in this case because we don't have any order expression here.

      How about add a check on

      orderSpec.isEmpty
      

      at https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExec.scala#L141
      and throw an exception saying "no order expressions is specified"?

      Attachments

        Activity

          People

            Unassigned Unassigned
            weiluo_ren123 Weiluo Ren
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: