[SPARK-647] ConnectionManager.sendMessage may create too many unnecessary connections

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.1
• Fix Version/s: 0.6.2, 0.7.0
• Component/s: Spark Core
• Assignee: Shane Huang
• Reporter: Shane Huang
• Created: 07/Jan/13 22:59
• Updated: 10/Jan/13 17:54
• Resolved: 10/Jan/13 17:54

Description

In ConnectionManager, sendMessage creates a new SendingConnection when connection host key is not found in connectionsById. But there might be too many unnecessary connections created before connectionsById is updated in the connection-manager-thread.run. In out test ConnectionMangerTest fails on "connection reset by peer" or "timeout" when there're too many message sending threads. We should make connectionRequests a map to track the connections by key so that connections can be reused.