[IGNITE-10898] Exchange coordinator failover breaks in some cases when node filter is used

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.8
• Assignee: Dmitriy Govorukhin
• Reporter: Alexey Goncharuk
• Created: 11/Jan/19 14:20
• Updated: 02/Aug/19 11:27
• Resolved: 17/Feb/19 21:49

Description

Currently if a node does not pass cache node filter, we do not store this cache affinity on the node unless the node is coordinator. This, however, may fail in the following scenario:
1) A node passing node filter joins cluster
2) During the join coordinator fails, new coordinator is selected for which previous exchange is completed
3) Next coordinator attempts to fetch the affinity, and joining node resends partitions single message, but there are two problems here. First, exchange fast-reply does not wait for the new affinity initialization which results in IllegalStateException. Second, such an attempt to fetch affinity may lead either to deadlock or to incorrectly fetched affinity (basically, coordinator must be in consensus with other nodes passing node filter)

Test attached reproduces the issue.

I suggest to always calculate and keep affinity on all nodes, even ones not passing the filter. In this case, there will be no need to fetch and recalculate affinity (initCoordinatorCaches will go away.

Issue Links

• causes: IGNITE-11631 Server node with PDS and SSL fails on start with NPE
• links to: GitHub Pull Request #5838
• links to: GitHub Pull Request #6004