[SPARK-40649] Infer date type for Json schema inference

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.2.1
• Component/s: SQL
• Assignee: Jonathan Cui
• Reporter: Xiaonan Yang
• Created: 04/Oct/22 18:18
• Updated: 12/Oct/22 17:46
• Resolved: 04/Oct/22 18:18

Description

1. If a column contains only dates, it should be of “date” type in the inferred schema

• If the date format and the timestamp format are identical (e.g. both are yyyy/mm/dd), entries will default to being interpreted as Date

2. If a column contains dates and timestamps, it should be of “timestamp” type in the inferred schema

 

A similar issue was opened in the past but was reverted due to the lack of strict pattern matching. 

Issue Links

• is a clone of: SPARK-39469 Infer date type for CSV schema inference
• is duplicated by: SPARK-40650 Infer date type for Json schema inference