[SPARK-2331] SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 2.0.0
• Component/s: Spark Core
• Target Version/s: 2.0.0
• Assignee: Reynold Xin
• Reporter: Ian Hummel
• Created: 30/Jun/14 17:41
• Updated: 21/Dec/15 22:11
• Resolved: 21/Dec/15 22:11

Description

The return type for SparkContext.emptyRDD is EmptyRDD[T].

It should be RDD[T]. That means you have to add extra type annotations on code like the below (which creates a union of RDDs over some subset of paths in a folder)

val rdds = Seq("a", "b", "c").foldLeft[RDD[String]](sc.emptyRDD[String]) { (rdd, path) ⇒
          rdd.union(sc.textFile(path))
        }

Issue Links

• links to: [Github] Pull Request #10394 (rxin)