[SPARK-3280] Made sort-based shuffle the default implementation

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: Shuffle, Spark Core
• Target Version/s: 1.2.0
• Assignee: Reynold Xin
• Reporter: Reynold Xin
• Created: 28/Aug/14 07:54
• Updated: 11/Sep/14 16:10
• Resolved: 08/Sep/14 03:42

Description

sort-based shuffle has lower memory usage and seems to outperform hash-based in almost all of our testing.

Issue Links

• is related to: SPARK-3385 Improve shuffle performance
• links to: [Github] Pull Request #2178 (rxin)