[IGNITE-11624] Discovery SPI: The client can handle events from the previous cluster after reconnect.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8
• Assignee: Nikita Amelchev
• Reporter: Nikita Amelchev
• Created: 25/Mar/19 11:39
• Updated: 10/Sep/19 15:25
• Resolved: 10/Sep/19 15:25

Description

Discovery has a queue for events. It's processed by event thread. If we hold up event processing using a listener on the client side and restarts cluster - the client will reconnect. After it reconnects it will continue processing events from the previous cluster.

This behavior produces bugs in MvccProcessor (IGNITE-11460) and hanging of partitions exchange on the client side. The reason is that discovery notifies components about reconnection in the notifier thread by calling the 'onLocalJoin' method. After it (or at the same time), components can process events from the previous cluster in their listeners and break their logic.

The order of events is fine - after processing previous cluster events - it will process client disconnection/reconnection and new cluster events.

The possible solution is to fix discovery logic. Make a guarantee that no one event from the previous cluster will be processed after the client reconnect ('onLocalJoin' called). For example, wait for the client disconnect event will be processed in the discovery event thread. Then start attempt to reconnect.

Dev-list discussion. Reproducer.

Issue Links

• relates to: IGNITE-11460 MVCC: Possible race on coordinator changing on client reconnection.
• links to: GitHub Pull Request #6365
• links to: Upsource