Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-7039

JdbcRdd doesn't support java.sql.Types.NVARCHAR

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.3.1
    • 1.3.2, 1.4.0
    • SQL
    • None
    • Spark 1.3.1

    Description

      When create a DataFrame from jdbc method through SqlContext:

      DataFrame df = sql.jdbc(url, fullTableName);
      

      If there is column type NVARCHAR, below exception will be thrown:

      Caused by: java.sql.SQLException: Unsupported type -9
      	at org.apache.spark.sql.jdbc.JDBCRDD$.getCatalystType(JDBCRDD.scala:78)
      	at org.apache.spark.sql.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:112)
      	at org.apache.spark.sql.jdbc.JDBCRelation.<init>(JDBCRelation.scala:133)
      	at org.apache.spark.sql.SQLContext.jdbc(SQLContext.scala:900)
      	at org.apache.spark.sql.SQLContext.jdbc(SQLContext.scala:852)
      

      When comparing the code between JDBCRDD.scala and java.sql.Types.java, the only type is not supported in JDBCRDD.scala is NVARCHAR. Because NCHAR is supported, so I think this is just a small mistake that people skip this type instead of ignore it intentionally.

      Attachments

        Activity

          People

            szheng79 Shuai Zheng
            szheng79 Shuai Zheng
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: