[SPARK-38284] Add the TIMESTAMPDIFF() function

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: SQL
• Epic Link: New functions in Spark SQL 3.3
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 22/Feb/22 08:40
• Updated: 04/Apr/22 09:19
• Resolved: 01/Mar/22 15:09

Description

Two functions that are part of the ODBC api and implemented virtually by ALL other databases are missing in Spark SQL.

TIMESTAMPADD() and TIMESTAMPDIFF()

See

• Time, Date, and Interval Functions
• TIMESTAMPDIFF function (ODBC compatible)

The TIMESTAMPDIFF() function is also VERY convenient and would be a great addition because getting the difference between two timestamps in seconds in spark today requires converting to unixtimestamp and back.

SELECT TIMESTAMPDIFF(SECOND, '2021-12-30 01:02:00':timestamp,  '2021-12-30 01:00:00':timestamp)
>120
-- Current Support
SELECT unix_timestamp('2021-12-30 01:00:00':timestamp) - unix_timestamp('2021-12-30 01:00:00':timestamp)
> 120

Issue Links

• is a parent of: SPARK-38535 Add the datetimeUnit enum to the grammar and use it in TIMESTAMPADD/DIFF
• links to: [Github] Pull Request #35607 (MaxGekk)
• links to: [Github] Pull Request #35607 (MaxGekk)