[SPARK-31980] Spark sequence() fails if start and end of range are identical dates

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.4.5, 2.4.6
• Fix Version/s: 2.4.7, 3.0.1, 3.1.0
• Component/s: SQL

Environment

Spark 2.4.4 standalone and on AWS EMR

• Assignee: JinxinTang
• Reporter: Dave DeCaprio
• Created: 12/Jun/20 20:45
• Updated: 20/Jun/20 07:40
• Resolved: 20/Jun/20 02:25

Description

 

The following Spark SQL query throws an exception

select sequence(cast("2011-03-01" as date), cast("2011-03-01" as date), interval 1 month)

The error is:

 

 

java.lang.ArrayIndexOutOfBoundsException: 1java.lang.ArrayIndexOutOfBoundsException: 1 at scala.runtime.ScalaRunTime$.array_update(ScalaRunTime.scala:92) at org.apache.spark.sql.catalyst.expressions.Sequence$TemporalSequenceImpl.eval(collectionOperations.scala:2681) at org.apache.spark.sql.catalyst.expressions.Sequence.eval(collectionOperations.scala:2514) at org.apache.spark.sql.catalyst.expressions.UnaryExpression.eval(Expression.scala:389)

 

Issue Links

• links to: [Github] Pull Request #28819 (TJX2014)
• links to: [Github] Pull Request #28819 (TJX2014)
• links to: [Github] Pull Request #28877 (TJX2014)