[SPARK-20334] Return a better error message when correlated predicates contain aggregate expression that has mixture of outer and local references

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Dilip Biswal
• Reporter: Dilip Biswal
• Created: 14/Apr/17 05:09
• Updated: 20/Apr/17 20:36
• Resolved: 20/Apr/17 20:36

Description

Currently subqueries with correlated predicates containing aggregate expression having mixture of outer references and local references generate a code gen error like following :

Cannot evaluate expression: min((input[0, int, false] + input[4, int, false]))
	at org.apache.spark.sql.catalyst.expressions.Unevaluable$class.doGenCode(Expression.scala:226)
	at org.apache.spark.sql.catalyst.expressions.aggregate.AggregateExpression.doGenCode(interfaces.scala:87)
	at org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:106)
	at org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:103)
	at scala.Option.getOrElse(Option.scala:121)
	at org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:103)
	at org.apache.spark.sql.catalyst.expressions.BinaryExpression.nullSafeCodeGen(Expression.scala:461)
	at org.apache.spark.sql.catalyst.expressions.BinaryExpression.defineCodeGen(Expression.scala:443)
	at org.apache.spark.sql.catalyst.expressions.BinaryComparison.doGenCode(predicates.scala:431)
	at org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:106)
	at org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:103)
	at scala.Option.getOrElse(Option.scala:121)
	at org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:103)

We should catch this situation and return a better error message to the user.

Issue Links

• links to: [Github] Pull Request #17636 (dilipbiswal)