[SPARK-5860] JdbcRDD: overflow on large range with high number of partitions

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.2.1
• Fix Version/s: 1.4.0
• Component/s: Spark Core
• Assignee: Jeroen Warmerdam
• Reporter: Jeroen Warmerdam
• Created: 17/Feb/15 10:38
• Updated: 21/Feb/15 20:43
• Resolved: 21/Feb/15 20:40

Description

val jdbcRDD = new JdbcRDD(sc, () =>
      DriverManager.getConnection(url, username, password),
      "SELECT id FROM documents WHERE ? <= id AND id <= ?",
      lowerBound = 1131544775L,
      upperBound = 567279358897692673L,
      numPartitions = 20,
      mapRow = r => (r.getLong("id"))
)

Issue Links

• links to: [Github] Pull Request #4701 (hotou)