Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-44044

Improve Error message for SQL Window functions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • 3.5.0
    • 3.5.0
    • Structured Streaming
    • None

    Description

      Right now, if window spec is used with a stream query, the error message looks like following:
      Non-time-based windows are not supported on streaming DataFrames/Datasets;
      Window [... <Sub Logic Plan>
      The message isn't very helpful to identify what's the problem is and some customers and even support engineers got confused by this. It is suggested that we call out aggregation function over the window spec so that the users can locate the part of the query that caused the problem easier.

      Attachments

        Activity

          People

            siying Siying Dong
            siying Siying Dong
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: