Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3587

Spark SQL can't support lead() over() window function

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 1.0.2, 1.1.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None
    • Environment:

      operator system is suse 11.3, three node ,every node has 48GB MEM and 16 CPU

      Description

      When run the following SQL:
      "select c.plateno, c.platetype,c.bay_id as start_bay_id,c.pastkkdate as start_time,lead(c.bay_id) over(partition by c.plateno order by c.pastkkdate) as end_bay_id, lead(c.pastkkdate) over(partition by c.plateno order by c.pastkkdate) as end_time from bay_car_test1 c"

      ----in HIVE, it is OK,

      -----but run in Spark SQL with HiveContext, it is error with:

      java.lang.RuntimeException:
      Unsupported language features in query: select c.plateno, c.platetype,c.bay_id as start_bay_id,c.pastkkdate as start_time,lead(c.bay_id) over(partition by c.plateno order by c.pastkkdate) as end_bay_id, lead(c.pastkkdate) over(partition by c.plateno order by c.pastkkdate) as end_time from bay_car_test1 c
      TOK_QUERY

      is Spark SQLcan't support lead() and over() function?

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              caoli caoli

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 504h
                504h
                Remaining:
                Remaining Estimate - 504h
                504h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Issue deployment