[SPARK-49583] datetime formatter violates assert of variable seconds fraction

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 10/Sep/24 08:17
• Updated: 10/Sep/24 13:48
• Resolved: 10/Sep/24 13:48

Description

The datetime pattern "\nSSSS\r" cause failure on assert while parsing the pattern by datetime formatter:

java.lang.AssertionError: assertion failed
	at scala.Predef$.assert(Predef.scala:264)
	at org.apache.spark.ErrorClassesJsonReader.getMessageTemplate(ErrorClassesJSONReader.scala:91)
	at org.apache.spark.ErrorClassesJsonReader.getErrorMessage(ErrorClassesJSONReader.scala:46)
	at org.apache.spark.SparkThrowableHelper$.getMessage(SparkThrowableHelper.scala:56)
	at org.apache.spark.SparkIllegalArgumentException.<init>(SparkException.scala:405)
	at org.apache.spark.SparkIllegalArgumentException.<init>(SparkException.scala:417)
	at org.apache.spark.SparkIllegalArgumentException.<init>(SparkException.scala:422)
	at org.apache.spark.sql.catalyst.util.DateTimeFormatterHelper$.$anonfun$createBuilderWithVarLengthSecondFraction$1(DateTimeFormatterHelper.scala:247)

Issue Links

• links to: GitHub Pull Request #48058