[SPARK-48081] Fix ClassCastException in NTile.checkInputDataTypes() when argument is non-foldable or of wrong type

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0, 3.5.2, 3.4.4
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Josh Rosen
• Reporter: Josh Rosen
• Created: 02/May/24 07:29
• Updated: 02/May/24 19:57
• Resolved: 02/May/24 14:23

Description

sql("select ntile(99.9) OVER (order by id) from range(10)")

results in

 java.lang.ClassCastException: class org.apache.spark.sql.types.Decimal cannot be cast to class java.lang.Integer (org.apache.spark.sql.types.Decimal is in unnamed module of loader 'app'; java.lang.Integer is in module java.base of loader 'bootstrap')
  at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:99)
  at org.apache.spark.sql.catalyst.expressions.NTile.checkInputDataTypes(windowExpressions.scala:877)
  at org.apache.spark.sql.catalyst.expressions.Expression.resolved$lzycompute(Expression.scala:267)
  at org.apache.spark.sql.catalyst.expressions.Expression.resolved(Expression.scala:267)
  at org.apache.spark.sql.catalyst.expressions.Expression.$anonfun$childrenResolved$1(Expression.scala:279)
  at org.apache.spark.sql.catalyst.expressions.Expression.$anonfun$childrenResolved$1$adapted(Expression.scala:279)
  at scala.collection.IterableOnceOps.forall(IterableOnce.scala:633)
  at scala.collection.IterableOnceOps.forall$(IterableOnce.scala:630)
  at scala.collection.AbstractIterable.forall(Iterable.scala:935)
  at org.apache.spark.sql.catalyst.expressions.Expression.childrenResolved(Expression.scala:279)
  at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$22$$anonfun$applyOrElse$157.applyOrElse(Analyzer.scala:2243) 

instead of the intended user-facing error message. This is a minor bug that was introduced in a previous error class refactoring PR.

Issue Links

• links to: GitHub Pull Request #46333
• links to: GitHub Pull Request #46336
• links to: GitHub Pull Request #46337