[SPARK-18141] jdbc datasource read fails when quoted columns (eg:mixed case, reserved words) in source table are used in the filter.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0, 2.0.1
• Fix Version/s: 2.1.0
• Component/s: SQL
• Assignee: Suresh Thalamati
• Reporter: Suresh Thalamati
• Created: 27/Oct/16 18:16
• Updated: 07/Jun/17 08:20
• Resolved: 02/Dec/16 03:14

Description

create table t1("Name" text, "Id" integer)
insert into t1 values('Mike', 1)

val df = sqlContext.read.jdbc(jdbcUrl, "t1", new Properties)

df.filter("Id = 1").show()

df.filter("`Id` = 1").show()

Error :
Cause: org.postgresql.util.PSQLException: ERROR: column "id" does not exist
Position: 35
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2182)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1911)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:173)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:622)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:472)
at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:386)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD.compute(JDBCRDD.scala:295)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)

I am working on fix for this issue, will submit PR soon.

Issue Links

• is duplicated by: SPARK-14460 DataFrameWriter JDBC doesn't Quote/Escape column names
• links to: [Github] Pull Request #15662 (sureshthalamati)