[SPARK-669] Send back task results through BlockManager instead of Akka messages

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.2, 0.7.0
• Fix Version/s: 0.8.1
• Assignee: Kay Ousterhout
• Reporter: Matei Alexandru Zaharia
• Created: 27/Jan/13 22:49
• Updated: 04/Oct/13 15:11
• Resolved: 04/Oct/13 15:11

Description

A common problem for users is that their task results are multiple MB in size, and Akka cannot send messages larger than its frameSize. It would be better to avoid this altogether by sending them through the BlockManager. The driver would then delete the result from the remote node after it fetched it.

Issue Links

• is related to: SPARK-747 Throw an exception on slaves if a message is sent that is larger than akka's max frame size