[SPARK-2242] Running sc.parallelize(..).count() hangs pyspark

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.1.0
• Component/s: PySpark
• Target Version/s: 1.1.0
• Assignee: Andrew Or
• Reporter: Xiangrui Meng
• Created: 23/Jun/14 03:30
• Updated: 05/Nov/14 10:45
• Resolved: 25/Jun/14 17:48

Description

Running the following code hangs pyspark in a shell:

sc.parallelize(range(100), 100).count()

It happens in the master branch, but not branch-1.0. And it seems that it only happens in a pyspark shell. andrewor14 helped confirm this bug.

Issue Links

• duplicates: SPARK-2244 pyspark - RDD action hangs (after previously succeeding)
• incorporates: SPARK-1850 Bad exception if multiple jars exist when running PySpark
• is duplicated by: SPARK-2244 pyspark - RDD action hangs (after previously succeeding)