[SPARK-24695] Move `CalendarInterval` to org.apache.spark.sql.types package

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.3.1
• Component/s: SQL
• Labels: bulk-closed
• Flags: Important
• Assignee: Unassigned
• Reporter: Priyanka Garg
• Created: 29/Jun/18 16:48
• Updated: 25/May/21 01:52
• Resolved: 25/May/21 01:38

Description

When i am trying to write an udf which returns calendar interval type, i am getting the following error:

Schema for type org.apache.spark.unsafe.types.CalendarInterval is not supported
java.lang.UnsupportedOperationException: Schema for type org.apache.spark.unsafe.types.CalendarInterval is not supported
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$schemaFor$1.apply(ScalaReflection.scala:781)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$schemaFor$1.apply(ScalaReflection.scala:715)
at scala.reflect.internal.tpe.TypeConstraints$UndoLog.undo(TypeConstraints.scala:56)
at org.apache.spark.sql.catalyst.ScalaReflection$class.cleanUpReflectionObjects(ScalaReflection.scala:825)
...

 

 

Issue Links

• blocks: SPARK-28492 Expose and support calendar interval type in PySpark
• blocks: SPARK-28493 Expose and support calendar interval type in SparkR
• blocks: SPARK-28491 Expose and support calendar interval type in SparkSQL
• links to: [Github] Pull Request #21679 (priyankagargnitk)
• links to: GitHub Pull Request #21679
• links to: GitHub Pull Request #25022