[SPARK-41452] to_char throws NullPointerException when format is null

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Assignee: Bruce Robbins
• Reporter: Bruce Robbins
• Created: 08/Dec/22 21:20
• Updated: 09/Dec/22 00:15
• Resolved: 09/Dec/22 00:15

Description

Example:

spark-sql> select to_char(454, null);
[INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace.
org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace.
...
Caused by: java.lang.NullPointerException
	at org.apache.spark.sql.catalyst.expressions.ToCharacter.numberFormat$lzycompute(numberFormatExpressions.scala:227)
	at org.apache.spark.sql.catalyst.expressions.ToCharacter.numberFormat(numberFormatExpressions.scala:227)
	at org.apache.spark.sql.catalyst.expressions.ToCharacter.numberFormatter$lzycompute(numberFormatExpressions.scala:228)
	at org.apache.spark.sql.catalyst.expressions.ToCharacter.numberFormatter(numberFormatExpressions.scala:228)
	at org.apache.spark.sql.catalyst.expressions.ToCharacter.checkInputDataTypes(numberFormatExpressions.scala:236)

Compare to to_binary:

spark-sql> SELECT to_binary('abc', null);
NULL
Time taken: 3.097 seconds, Fetched 1 row(s)
spark-sql>

Also compare to to_char in PostgreSQL 14.6:

select to_char(454, null) is null as to_char_is_null;

 to_char_is_null 
-----------------
 t
(1 row)

Issue Links

• links to: [Github] Pull Request #38986 (bersprockets)