[SPARK-17536] Minor performance improvement to JDBC batch inserts

• Type: Improvement
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.1.0
• Component/s: SQL
• Labels: perfomance
• Assignee: John Muller
• Reporter: John Muller
• Created: 14/Sep/16 15:01
• Updated: 15/Sep/16 09:01
• Resolved: 15/Sep/16 09:00

Description

JDBC batch inserts currently are set to repeatedly retrieve the number of fields inside the row iterator:

https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala#L598

val numFields = rddSchema.fields.length

This value does not change and can be set prior to the loop.

Issue Links

• links to: [Github] Pull Request #15098 (blue666man)