[SPARK-28015] Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.3, 2.0.2, 2.1.3, 2.2.3, 2.3.3, 2.4.3
• Fix Version/s: 2.4.4, 3.0.0
• Component/s: SQL
• Labels: correctness
• Assignee: Max Gekk
• Reporter: Yuming Wang
• Created: 12/Jun/19 07:07
• Updated: 11/Jul/19 08:48
• Resolved: 11/Jul/19 01:13

Description

Invalid date formats should throw an exception:

SELECT date '1999 08 01'
1999-01-01

Supported date formats:
https://github.com/apache/spark/blob/ab8710b57916a129fcb89464209361120d224535/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala#L365-L374

Since Spark 1.6.3 ~ 2.4.3, the behavior is the same.

spark-sql> SELECT CAST('1999 08 01' AS DATE);
1999-01-01

Hive returns NULL.

Issue Links

• links to: GitHub Pull Request #25097