[SPARK-41205] Bad error message when try_to_binary called with non-foldable format

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 3.3.1, 3.4.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Bruce Robbins
• Created: 19/Nov/22 22:31
• Updated: 23/Nov/22 17:12
• Resolved: 23/Nov/22 17:12

Description

For example:

spark-sql> SELECT try_to_binary(col1, col2) from values ('abc', 'utf-8') as data(col1, col2);
[INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace.
org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace.
	at org.apache.spark.SparkException$.internalError(SparkException.scala:88)
...
Caused by: java.lang.AssertionError: assertion failed
	at scala.Predef$.assert(Predef.scala:208)
	at org.apache.spark.sql.catalyst.expressions.ToBinary.$anonfun$replacement$1(stringExpressions.scala:2597)
	at scala.Option.map(Option.scala:230)
	at org.apache.spark.sql.catalyst.expressions.ToBinary.replacement$lzycompute(stringExpressions.scala:2596)
	at org.apache.spark.sql.catalyst.expressions.ToBinary.replacement(stringExpressions.scala:2596)
	at org.apache.spark.sql.catalyst.expressions.RuntimeReplaceable.dataType(Expression.scala:354)

Compare to to_binary:

spark-sql> SELECT to_binary(col1, col2) from values ('abc', 'utf-8') as data(col1, col2);
The 'format' parameter of function 'to_binary' needs to be a string literal.; line 1 pos 7
spark-sql> 

Issue Links

• duplicates: SPARK-41174 Propagate an error class to users for invalid `format` of `to_binary()`
• links to: [Github] Pull Request #38727 (bersprockets)