[SPARK-11230] Add Outbox to cache the sending messages to resolve the message disorder issue

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Shixiong Zhu
• Created: 21/Oct/15 11:13
• Updated: 21/Oct/15 11:17
• Resolved: 21/Oct/15 11:17

Description

The current NettyRpc has a message order issue because it uses a thread pool to send messages. E.g., running the following two lines,

ref.send("A")
ref.send("B")

The remote endpoint may see "B" before "A" because sending "A" and "B" are in parallel.

To resolve this issue, we need to add an outbox for each connection, and if we are connecting to the remote node, we can just cache the sending messages in the outbox and send them one by one when the connection is established.

Issue Links

• duplicates: SPARK-11098 RPC message ordering is not guaranteed