[SPARK-10217] Spark SQL cannot handle ordering directive in ORDER BY clauses with expressions

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 1.4.1
• Component/s: SQL
• Labels: SQL analyzers

Environment

Ubuntu on AWS

• Assignee: Unassigned
• Reporter: Simeon Simeonov
• Created: 25/Aug/15 04:11
• Updated: 15/Oct/15 17:03
• Resolved: 15/Oct/15 17:03

Description

Spark SQL supports expressions in ORDER BY clauses, e.g.,

scala> sqlContext.sql("select cnt from test order by (cnt + cnt)")
res2: org.apache.spark.sql.DataFrame = [cnt: bigint]

However, the analyzer gets confused when there is an explicit ordering directive (ASC/DESC):

scala> sqlContext.sql("select cnt from test order by (cnt + cnt) asc")
15/08/25 04:08:02 INFO ParseDriver: Parsing command: select cnt from test order by (cnt + cnt) asc
org.apache.spark.sql.AnalysisException: extraneous input 'asc' expecting EOF near '<EOF>'; line 1 pos 40
	at org.apache.spark.sql.hive.HiveQl$.createPlan(HiveQl.scala:289)
	at org.apache.spark.sql.hive.ExtendedHiveQlParser$$anonfun$hiveQl$1.apply(ExtendedHiveQlParser.scala:41)
	at org.apache.spark.sql.hive.ExtendedHiveQlParser$$anonfun$hiveQl$1.apply(ExtendedHiveQlParser.scala:40)
        ...