Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20557

JdbcUtils doesn't support java.sql.Types.TIMESTAMP_WITH_TIMEZONE

    XMLWordPrintableJSON

    Details

    • Target Version/s:
    • Flags:
      Patch

      Description

      Reading from an Oracle DB table with a column of type TIMESTAMP WITH TIME ZONE via jdbc (spark.sqlContext.read.format("jdbc").option(...).load()) results in an error:

      Unsupported type -101
      org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.org$apache$spark$sql$execution$datasources$jdbc$JdbcUtils$$getCatalystType(JdbcUtils.scala:209)
      org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$5.apply(JdbcUtils.scala:246)
      org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$5.apply(JdbcUtils.scala:246)

      That is because the type java.sql.Types.TIMESTAMP_WITH_TIMEZONE (in Java since 1.8) is missing in JdbcUtils.scala

      This is similar to SPARK-7039.

      I created a pull request with a fix.

        Attachments

          Activity

            People

            • Assignee:
              smilegator Xiao Li
              Reporter:
              JannikArndt Jannik Arndt
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 2h
                2h
                Remaining:
                Remaining Estimate - 2h
                2h
                Logged:
                Time Spent - Not Specified
                Not Specified