[FLINK-1063] Race condition in NettyConnectionManager

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6-incubating, 0.7.0-incubating
• Fix Version/s: 0.6.1-incubating, 0.7.0-incubating
• Component/s: Runtime / Coordination
• Assignee: Ufuk Celebi
• Reporter: Ufuk Celebi
• Created: 25/Aug/14 09:19
• Updated: 07/Sep/14 10:49
• Resolved: 07/Sep/14 10:40

Description

The TCP channel queuing mechanism in NettyConnectionManager has a race condition, which may result in re-ordering of envelopes at the receiver (the dreaded "Expected data packet X but received Y" exception).

Thanks to AHeise for reporting the problem.

The problem has been introduced with commits 52512636444902497e47ccbfb1cabaffb3e23343 ... 32d168f439bdb5dfab02a3ab2d12e87d0622a67e.

I will revert the respective commits and implement a fall back, which limits TCP channel multiplexing and immediately closes TCP channels.

Issue Links

• relates to: FLINK-1093 Add non-multiplexing NetworkConnectionManager