[HDDS-9321] LegacyReplicationManager: Save UNHEALTHY replicas with highest BCSID for a QUASI_CLOSED container

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: SCM
• Labels: pull-request-available
• Target Version/s: 1.4.0
• Assignee: Siddhant Sangwan
• Reporter: Siddhant Sangwan
• Created: 19/Sep/23 14:38
• Updated: 28/Sep/23 09:39
• Resolved: 28/Sep/23 09:39

Description

Mix of quasi-closed and unhealthy replicas blocks decommission even if sufficiently replicated.
a. Caused when only some of the replicas hit the error during write.
b. Can be fixed by removing this check:

if (!replicaSet.isHealthy()) {
          if (LOG.isDebugEnabled()) {
            unhealthyIDs.add(cid);
          }
          if (unhealthy < CONTAINER_DETAILS_LOGGING_LIMIT

However, simply removing that check is not a complete solution. We need to try and preserve any UNHEALTHY replicas that have the greatest Sequence ID. This jira focus on the Legacy Replication Manager part of the fix to preserve such UNHEALTHY replicas.

Issue Links

• is required by: HDDS-9354 LegacyReplicationManager: Unhealthy replicas of a sufficiently replicated container can block decommissioning
• requires: HDDS-9352 LegacyReplicationManager: Ignore any Datanodes that are not in-service and healthy when finding unique origins
• links to: GitHub Pull Request #5366