[SPARK-7746] SetFetchSize for JDBCRDD's prepareStatement

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0, 1.3.1
• Fix Version/s: 1.4.0
• Component/s: SQL
• Flags: Important
• Assignee: L. C. Hsieh
• Reporter: Paul Wu
• Created: 19/May/15 23:47
• Updated: 21/May/15 05:24
• Resolved: 21/May/15 05:24

Description

The prepareStatement created internal to compute() method in JDBCRDD should have some options that the setFetchSize() can be set. We found that the parameter can affect Oracle DB tremendously. With current implementation, we have no way to set the size.

One of my team did his implementation, and we found 10X improvement by setting the size to a proper number.

Issue Links

• links to: [Github] Pull Request #6283 (viirya)