[SPARK-33160] Allow saving/loading INT96 in parquet w/o rebasing

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: SQL
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 15/Oct/20 11:56
• Updated: 12/Dec/22 18:11
• Resolved: 20/Oct/20 05:59

Description

Currently, Spark always performs rebasing of INT96 columns in Parquet datasource but this is not required by parquet spec. This tickets aims to allow users to turn off rebasing via SQL config.

Issue Links

• is cloned by: SPARK-34440 Allow saving/loading datetime in ORC w/o rebasing
• is related to: SPARK-33210 Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default
• links to: [Github] Pull Request #30056 (MaxGekk)
• links to: [Github] Pull Request #30056 (MaxGekk)
• links to: [Github] Pull Request #30118 (MaxGekk)
• links to: [Github] Pull Request #30132 (MaxGekk)