[SPARK-5201] ParallelCollectionRDD.slice(seq, numSlices) has int overflow when dealing with inclusive range

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.2.1, 1.3.0
• Component/s: Spark Core
• Labels: rdd
• Target Version/s: 1.2.1, 1.3.0
• Assignee: YE
• Reporter: YE
• Created: 12/Jan/15 07:22
• Updated: 16/Jan/15 17:31
• Resolved: 16/Jan/15 17:31

Description

 sc.makeRDD(1 to (Int.MaxValue)).count       // result = 0
 sc.makeRDD(1 to (Int.MaxValue - 1)).count   // result = 2147483646 = Int.MaxValue - 1
 sc.makeRDD(1 until (Int.MaxValue)).count    // result = 2147483646 = Int.MaxValue - 1

More details on the discussion https://github.com/apache/spark/pull/2874

Issue Links

• links to: [Github] Pull Request #4002 (advancedxy)