[HBASE-23770] [Flakey Tests] TestRegionReplicasWithRestartScenarios#testWhenRestart

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.3.0
• Component/s: flakies
• Assignee: Michael Stack
• Reporter: Michael Stack
• Created: 30/Jan/20 06:09
• Updated: 07/Apr/20 21:39
• Resolved: 31/Jan/20 16:50

Description

Fails about 35% of the time in the GCE build. Let me attach a picture from current flakies dashboard for branch-2.

The test starts a cluster of three RS w/ 3 region replicas. It then stops a server, starts a new one, and then expects that the remaining three nodes do not have instances where two region replicas have landed on a single server.

It fails sporadically (reproducible locally) because when the SCP runs its assign, sometimes timing has it so Master knows of two servers only. Making the new start before the old one is stopped (instead of other way around) seems to fix the test – there'll be three servers up when SCP runs.