[SPARK-31880] Adjacent value parsing not supported for Localized Patterns because of JDK bug

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 3.0.0, 3.1.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Kent Yao 2
• Created: 01/Jun/20 02:55
• Updated: 12/Dec/22 18:11
• Resolved: 12/Jun/20 03:38

Description

spark-sql> set spark.sql.legacy.timeParserPolicy=exception;
spark.sql.legacy.timeParserPolicy	exception
spark-sql> select to_timestamp('202011', 'YYYYww');

Caused by: java.time.format.DateTimeParseException: Text '202011' could not be parsed at index 0
	at java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:1949)
	at java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1777)
	at org.apache.spark.sql.catalyst.util.Iso8601TimestampFormatter.$anonfun$parse$1(TimestampFormatter.scala:79)
	... 99 more

spark-sql> set spark.sql.legacy.timeParserPolicy=legacy;
spark.sql.legacy.timeParserPolicy	legacy
spark-sql> select to_timestamp('202011', 'YYYYwu');
2019-12-30 00:00:00
spark-sql> select to_timestamp('202011', 'YYYYww');
2020-03-08 00:00:00

The result could vary between different JDKs

Issue Links

• links to: JDK-8145633