[SPARK-4304] sortByKey() will fail on empty RDD

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.0.2, 1.1.0, 1.2.0
• Fix Version/s: 1.0.3, 1.1.1, 1.2.0
• Component/s: PySpark
• Target Version/s: 1.0.3, 1.1.1, 1.2.0
• Assignee: Davies Liu
• Reporter: Davies Liu
• Created: 08/Nov/14 01:09
• Updated: 08/Nov/14 04:58
• Resolved: 08/Nov/14 04:58

Description

>>> sc.parallelize(zip(range(4), range(0)), 5).sortByKey().count()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/davies/work/spark/python/pyspark/rdd.py", line 532, in sortByKey
    for i in range(0, numPartitions - 1)]
IndexError: list index out of range
>>>

Issue Links

• links to: [Github] Pull Request #3162 (davies)
• links to: [Github] Pull Request #3163 (davies)