[SPARK-44280] Add convertJavaTimestampToTimestamp in JDBCDialect API

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 3.5.0
• Component/s: Spark Core
• Labels: pull-request-available
• Assignee: Mingkang Li
• Reporter: Mingkang Li
• Created: 03/Jul/23 18:35
• Updated: 19/Mar/24 16:01
• Resolved: 02/Aug/23 06:48

Description

A new method, convertJavaTimestampToTimestamp, is introduced to the JDBCDialects API, providing the capability for JDBC dialects to override the default Java timestamp conversion behavior. This enhancement is particularly beneficial for databases such as PostgreSQL, which feature special values for timestamps representing positive and negative infinity. 

The pre-existing default behavior of timestamp conversion potentially triggers an overflow due to these special values (i.e. The executor would crash if you select a column that contains infinity timestamps in PostgreSQL.) By integrating this new function, we can mitigate such issues, enabling more versatile and robust timestamp value conversions across various JDBC-based connectors.

Issue Links

• links to: [Github] Pull Request #41843 (mingkangli-db)
• links to: GitHub Pull Request #41843