[SPARK-7311] Enable in-memory serialized map-side shuffle to work with SQL serializers

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.4.0
• Component/s: Spark Core, SQL
• Epic Link: Tungsten Phase 1
• Assignee: Josh Rosen
• Reporter: Sandy Ryza
• Created: 01/May/15 20:58
• Updated: 06/May/15 17:53
• Resolved: 06/May/15 17:53

Description

Right now it only works with KryoSerializer, but it would be useful to make it work with any Serializer that writes objects in a relocatable / self-contained fashion.

Issue Links

• blocks: SPARK-7081 Faster sort-based shuffle path using binary processing cache-aware sort
• links to: [Github] Pull Request #5924 (JoshRosen)