[SPARK-26246] Infer timestamp types from JSON

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 3.0.0
• Component/s: SQL
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 02/Dec/18 20:49
• Updated: 12/Dec/22 18:11
• Resolved: 18/Dec/18 05:51

Description

Currently, TimestampType cannot be inferred from JSON. To parse JSON string, you have to specify schema explicitly if JSON input contains timestamps. This ticket aims to extend JsonInferSchema to support such inferring.

Issue Links

• causes: SPARK-32130 Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4
• links to: [Github] Pull Request #23201 (MaxGekk)
• links to: GitHub Pull Request #23201
• links to: GitHub Pull Request #23455