[SPARK-21446] [SQL] JDBC Postgres fetchsize parameter ignored again

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.0, 2.1.1, 2.2.0
• Fix Version/s: 2.1.2, 2.2.1, 2.3.0
• Component/s: SQL
• Assignee: Zhang Chengbo
• Reporter: Zhang Chengbo
• Created: 18/Jul/17 00:01
• Updated: 19/Jul/17 21:48
• Resolved: 19/Jul/17 21:48

Description

This bug was fixed in https://github.com/apache/spark/pull/9861
but reappears in spark 2.1 +
The following code may not be executed in org.apache.spark.sql.jdbc.PostgresDialect because "properties" is "asConnectionProperties" but "asConnectionProperties" can not have "fetchsize"

if (properties.getOrElse(JDBCOptions.JDBC_BATCH_FETCH_SIZE, "0").toInt > 0) {
      connection.setAutoCommit(false)
}

Issue Links

• links to: [Github] Pull Request #18665 (DFFuture)