[IGNITE-5302] Empty LOST partition may be used as OWNING after resetting lost partitions

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Won't Fix
• Fix Version/s: 2.8
• Labels: MakeTeamcityGreenAgain Muted_test test-fail
• Assignee: Unassigned
• Reporter: Sergey Chugunov
• Created: 26/May/17 14:06
• Updated: 24/Sep/19 13:02
• Resolved: 24/Sep/19 13:02

Description

Notes

Test testPartitionLossAndRecover reproducing the issue can be found in ignite-5267 branch with PDS functionality.

Steps to reproduce

1. Four nodes are started, some key is added to partitioned cache
2. Primary and backup nodes for the key are stopped, key's partition is declared LOST on remaining nodes
3. Primary and backup nodes are started again, cache's lost partitions are reset
4. Key is requested from cache

Expected behavior

Correct value is returned from primary for this partition

Actual behavior

Request for value is sent to node where partition is empty (not to primary node), null is returned

Latest findings

1. The main problem with the scenario is that request for key gets mapped not only to P/B nodes with real value but also to the node where that partition existed only in LOST state after P/B shutdown on step #2
2. It was found that on step #3 after primary and backup are joined partition counter is increased for empty partition in LOST state which looks wrong

Issue Links

• is duplicated by: IGNITE-10058 resetLostPartitions() leaves an additional copy of a partition in the cluster
• is related to: IGNITE-7832 Ignite.resetLostPartitions() resets state under race.
• relates to: IGNITE-5792 IgnitePdsAtomicCacheRebalancingTest.testRebalancingOnRestartAfterCheckpoint is failing on TC