The test all data types test uses the legacy parser for dates/timestamps (https://github.com/apache/spark/pull/23196/files#diff-3986f801dd1335af3f300b006a03e987R129). The test is passed on UTC timezone and new parser but fails on different timezones (see):
== Correct Answer - 10 == == Spark Answer - 10 == ... ![10,1670-02-11 14:09:54.746] [10,1670-02-11 14:08:56.746] == Correct Answer - 10 == == Spark Answer - 10 == [1,6246-07-23 20:34:56.968] [1,6246-07-23 20:34:56.968] ![2,0109-07-20 18:38:03.788] [2,0109-07-20 18:37:05.788]
The ticket aims to switching on new parser independently from timezones.
- is related to
-
SPARK-26503 Get rid of spark.sql.legacy.timeParser.enabled
-
- Resolved
-
- links to