[SPARK-6878] Sum on empty RDD fails with exception

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.2.2, 1.3.2, 1.4.0
• Component/s: Spark Core
• Flags: Patch
• Assignee: Erik van Oosten
• Reporter: Erik van Oosten
• Created: 13/Apr/15 10:56
• Updated: 15/Jun/15 13:01
• Resolved: 14/Apr/15 11:40

Description

Sum on an empty RDD throws an exception. Expected result is 0.

A simple fix is the replace

class DoubleRDDFunctions {
  def sum(): Double = self.reduce(_ + _)

with:

class DoubleRDDFunctions {
  def sum(): Double = self.aggregate(0.0)(_ + _, _ + _)

Issue Links

• is related to: SPARK-8373 When an RDD has no partition, Python sum will throw "Can not reduce() empty RDD"
• links to: [Github] Pull Request #5489 (erikvanoosten)