[SPARK-15379] String to date and timestamp conversion should return null on some invalid date string

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.6.1, 2.0.0
• Fix Version/s: 2.0.0
• Component/s: SQL
• Assignee: Yang Wang
• Reporter: Yang Wang
• Created: 18/May/16 08:18
• Updated: 27/Sep/16 09:15
• Resolved: 23/May/16 02:31

Description

spark.sql("select cast('2016-02-30 00:00:00' as date)") returns 2016-03-01, but spark.sql("select cast('2016-02-32 00:00:00' as date)") returns null. (See this notebook: https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/7973071962862063/3036006015533946/58107563000366/latest.html)
I suppose all invalid dates should be returned as null.
'2016-04-31', '2016-06-31', etc. also have the same issue.

Issue Links

• is related to: SPARK-12045 Use joda's DateTime to replace Calendar
• links to: [Github] Pull Request #13169 (wangyang1992)