[SPARK-7078] Cache-aware binary processing in-memory sort

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: Shuffle, Spark Core
• Target Version/s: 1.5.0
• Assignee: Josh Rosen
• Reporter: Reynold Xin
• Created: 23/Apr/15 07:32
• Updated: 17/May/20 18:29
• Resolved: 10/Jul/15 23:46

Description

A cache-friendly sort algorithm that can be used eventually for:

• sort-merge join
• shuffle

See the old alpha sort paper: http://research.microsoft.com/pubs/68249/alphasort.doc

Note that state-of-the-art for sorting has improved quite a bit, but we can easily optimize the sorting algorithm itself later.

Issue Links

• blocks: SPARK-7079 Cache-aware external sort
• is blocked by: SPARK-8932 Support copy in UnsafeRow as long as ObjectPool is not used
• links to: [Github] Pull Request #6444 (JoshRosen)