[SPARK-1240] takeSample called on empty RDD never ends

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.9.1, 1.0.0
• Assignee: Nan Zhu
• Reporter: Jaroslav Kamenik
• Created: 13/Mar/14 03:04
• Updated: 16/Mar/14 22:38
• Resolved: 16/Mar/14 22:18

Description

It seems, that method takeSample ends in infinite loop if called on empty RDD, trying to collect enough samples.

val list = List[String]("aaa")

val rdd = sc.parallelize(list)
rdd.takeSample(true, 1, System.nanoTime.toInt)

val empty = rdd.filter(_ => false)
empty.takeSample(true, 1, System.nanoTime.toInt)