[SPARK-47696] try_to_timestamp should handle SparkUpgradeException

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 4.0.0, 3.5.2, 3.4.3
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Gengliang Wang
• Reporter: Gengliang Wang
• Created: 02/Apr/24 17:56
• Updated: 26/Apr/24 22:45
• Resolved: 26/Apr/24 22:45

Description

Currently, try_to_timestamp will throw an exception on legacy timestamp input.

> SELECT try_to_timestamp('2016-12-1', 'yyyy-MM-dd')

org.apache.spark.SparkUpgradeException: [INCONSISTENT_BEHAVIOR_CROSS_VERSION.PARSE_DATETIME_BY_NEW_PARSER] You may get a different result due to the upgrading to Spark >= 3.0:
Fail to parse '2016-12-1' in the new parser.
You can set "spark.sql.legacy.timeParserPolicy" to "LEGACY" to restore the behavior before Spark 3.0, or set to "CORRECTED" and treat it as an invalid datetime string. SQLSTATE: 42K0B 

It should return null instead of error.

Issue Links

• links to: GitHub Pull Request #45822