[SPARK-47447] Allow reading Parquet TimestampLTZ as TimestampNTZ

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Gengliang Wang
• Reporter: Gengliang Wang
• Created: 18/Mar/24 18:54
• Updated: 20/Mar/24 06:05
• Resolved: 20/Mar/24 06:05

Description

Currently, Parquet TimestampNTZ type columns can be read as TimestampLTZ, while reading TimestampLTZ as TimestampNTZ will cause errors. This makes it impossible to read parquet files containing both TimestampLTZ and TimestampNTZ as TimestampNTZ.

To make the data type system on Parquet simpler, we should allow reading TimestampLTZ as TimestampNTZ in the Parquet data source.

Issue Links

• links to: GitHub Pull Request #45571