[SPARK-3074] support groupByKey() with hot keys in PySpark

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: PySpark
• Assignee: Davies Liu
• Reporter: Davies Liu
• Created: 15/Aug/14 16:21
• Updated: 07/Oct/16 23:32
• Resolved: 10/Apr/15 00:07

Issue Links

• contains: SPARK-2999 Compress all the serialized data
• is related to: SPARK-1823 ExternalAppendOnlyMap can still OOM if one key is very large
• links to: [Github] Pull Request #1977 (davies)