Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11623

Sparksql-1.4.1 DataFrameWrite.jdbc() bug

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 1.4.1, 1.5.1
    • Fix Version/s: None
    • Component/s: Java API, Spark Submit, SQL
    • Labels:
      None
    • Environment:

      Spark stand alone cluster

      Description

      I am running spark-submit in window 8.1 with spark standalone cluster (01 worker and 01 master), the job throw Exception in DataFrameWrite.jdbc(..) scala function.
      We found that the following test:
      var tableExists = JdbcUtils.tableExists(conn, table)
      always return false event if we already created a table.
      That drive the function to do creating table from specified DataFrame and the SQL Syntax error for creating table, we locate the SQL execution statement hereafter:
      if (!tableExists)

      { val schema = JDBCWriteDetails.schemaString(df, url) val sql = s"CREATE TABLE $table ($schema)" conn.prepareStatement(sql).executeUpdate() // This execution cause sql syntax error }

      This happened with spark-1.4.1 and Spark-1.5.1 (our dev environment).
      Please help!

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                nghia.n.v2007@gmail.com Nguyen Van Nghia
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: