[SPARK-20557] JdbcUtils doesn't support java.sql.Types.TIMESTAMP_WITH_TIMEZONE

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.0, 2.3.0
• Fix Version/s: 2.3.0
• Component/s: SQL
• Labels: easyfix jdbc oracle sql timestamp
• Target Version/s: 2.3.0
• Flags: Patch
• Assignee: Xiao Li
• Reporter: Jannik Arndt
• Created: 02/May/17 13:01
• Updated: 12/Dec/17 00:33
• Resolved: 12/Dec/17 00:33

Description

Reading from an Oracle DB table with a column of type TIMESTAMP WITH TIME ZONE via jdbc (spark.sqlContext.read.format("jdbc").option(...).load()) results in an error:

Unsupported type -101
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.org$apache$spark$sql$execution$datasources$jdbc$JdbcUtils$$getCatalystType(JdbcUtils.scala:209)
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$5.apply(JdbcUtils.scala:246)
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$5.apply(JdbcUtils.scala:246)

That is because the type java.sql.Types.TIMESTAMP_WITH_TIMEZONE (in Java since 1.8) is missing in JdbcUtils.scala

This is similar to SPARK-7039.

I created a pull request with a fix.

Issue Links

• links to: [Github] Pull Request #17832 (JannikArndt)
• links to: [Github] Pull Request #17835 (gatorsmile)
• links to: [Github] Pull Request #19939 (gatorsmile)