Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16625

Oracle JDBC table creation fails with ORA-00902: invalid datatype

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.6.2
    • 2.1.0, 2.1.2
    • SQL
    • None

    Description

      Unfortunately I know very little about databases, but I figure this is a bug.

      I have a DataFrame with the following schema:

      StructType(StructField(dst,StringType,true), StructField(id,LongType,true), StructField(src,StringType,true))
      

      I am trying to write it to an Oracle database like this:

      String url = "jdbc:oracle:thin:root/rootroot@<ip address>:1521:db";
      java.util.Properties p = new java.util.Properties();
      p.setProperty("driver", "oracle.jdbc.OracleDriver");
      df.write().mode("overwrite").jdbc(url, "my_table", p);
      

      And I get:

      Exception in thread "main" java.sql.SQLSyntaxErrorException: ORA-00902: invalid datatype
      
      	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:461)
      	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:402)
      	at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1108)
      	at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:541)
      	at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:264)
      	at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:598)
      	at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:213)
      	at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:26)
      	at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:1241)
      	at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1558)
      	at oracle.jdbc.driver.OracleStatement.executeUpdateInternal(OracleStatement.java:2498)
      	at oracle.jdbc.driver.OracleStatement.executeUpdate(OracleStatement.java:2431)
      	at oracle.jdbc.driver.OracleStatementWrapper.executeUpdate(OracleStatementWrapper.java:975)
      	at org.apache.spark.sql.DataFrameWriter.jdbc(DataFrameWriter.scala:302)
      

      The Oracle server I am running against is the one I get on Amazon RDS for engine type oracle-se. The same code (with the right driver) against the RDS instance with engine type MySQL works.

      The error message is the same as in https://issues.apache.org/jira/browse/SPARK-12941. Could it be that Long is also translated into the wrong data type? Thanks.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            yumwang Yuming Wang
            darabos Daniel Darabos
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment