[SPARK-33163] Check the metadata key 'org.apache.spark.legacyDateTime' in Avro/Parquet files

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.0.2, 3.1.0
• Component/s: SQL
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 15/Oct/20 18:57
• Updated: 12/Dec/22 18:11
• Resolved: 16/Oct/20 01:29

Description

Add tests to check that the metadata key 'org.apache.spark.legacyDateTime' is saved correctly depending on the SQL configs:

1. spark.sql.legacy.avro.datetimeRebaseModeInWrite
2. spark.sql.legacy.parquet.datetimeRebaseModeInWrite

Issue Links

• links to: [Github] Pull Request #30061 (MaxGekk)
• links to: [Github] Pull Request #30061 (MaxGekk)
• links to: [Github] Pull Request #31396 (MaxGekk)