[SPARK-12941] Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.1
• Fix Version/s: 1.4.2, 1.5.3, 1.6.2, 2.0.0
• Component/s: Spark Core

Environment

Apache Spark 1.4.2.2

• Assignee: Thomas Sebastian
• Reporter: Jose Martinez Poblete
• Created: 20/Jan/16 22:34
• Updated: 03/Mar/16 07:40
• Resolved: 26/Feb/16 06:52

Description

When exporting data from Spark to Oracle, string datatypes are translated to TEXT for Oracle, this is leading to the following error

java.sql.SQLSyntaxErrorException: ORA-00902: invalid datatype

As per the following code:

https://github.com/apache/spark/blob/branch-1.4/sql/core/src/main/scala/org/apache/spark/sql/jdbc/jdbc.scala#L144

See also:

http://stackoverflow.com/questions/31287182/writing-to-oracle-database-using-apache-spark-1-4-0

Issue Links

• links to: [Github] Pull Request #10912 (thomastechs)
• links to: [Github] Pull Request #11306 (thomastechs)
• links to: [Github] Pull Request #11489 (thomastechs)