[SPARK-45106] percentile_cont gets internal error when user input fails runtime replacement's input type check

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.2, 3.4.1, 3.5.0, 4.0.0
• Fix Version/s: 3.5.1
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Bruce Robbins
• Reporter: Bruce Robbins
• Created: 08/Sep/23 14:19
• Updated: 08/Sep/23 19:40
• Resolved: 08/Sep/23 19:40

Description

This query throws an internal error rather than producing a useful error message:

select percentile_cont(b) WITHIN GROUP (ORDER BY a DESC) as x 
from (values (12, 0.25), (13, 0.25), (22, 0.25)) as (a, b);

[INTERNAL_ERROR] Cannot resolve the runtime replaceable expression "percentile_cont(a, b)". The replacement is unresolved: "percentile(a, b, 1)".
org.apache.spark.SparkException: [INTERNAL_ERROR] Cannot resolve the runtime replaceable expression "percentile_cont(a, b)". The replacement is unresolved: "percentile(a, b, 1)".
	at org.apache.spark.SparkException$.internalError(SparkException.scala:92)
	at org.apache.spark.SparkException$.internalError(SparkException.scala:96)
	at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis0$6(CheckAnalysis.scala:313)
	at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis0$6$adapted(CheckAnalysis.scala:277)
...

It should instead inform the user that the input expression must be foldable.

PercentileCont does not check the user's input. If the runtime replacement (an instance of Percentile) rejects the user's input, the runtime replacement ends up unresolved.

Issue Links

• links to: GitHub Pull Request #42857