[HDDS-9354] LegacyReplicationManager: Unhealthy replicas of a sufficiently replicated container can block decommissioning

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: SCM
• Labels: pull-request-available
• Target Version/s: 1.4.0
• Assignee: Siddhant Sangwan
• Reporter: Siddhant Sangwan
• Created: 26/Sep/23 19:10
• Updated: 14/Dec/23 08:34
• Resolved: 06/Oct/23 04:27

Description

Mix of quasi-closed and unhealthy replicas blocks decommission even if sufficiently replicated.
a. Caused when only some of the replicas hit the error during write.
b. Can be fixed by removing this check:

if (!replicaSet.isHealthy()) {
          if (LOG.isDebugEnabled()) {
            unhealthyIDs.add(cid);
          }
          if (unhealthy < CONTAINER_DETAILS_LOGGING_LIMIT

However, simply removing that check is not a complete solution. We need to try and preserve any UNHEALTHY replicas that have the greatest Sequence ID. https://issues.apache.org/jira/browse/HDDS-9321 takes care of the Legacy Replication Manager side of things to preserve such UNHEALTHY replicas. This jira focuses on the Decommissioning side.

Issue Links

• requires: HDDS-9321 LegacyReplicationManager: Save UNHEALTHY replicas with highest BCSID for a QUASI_CLOSED container
• links to: GitHub Pull Request #5369
• links to: GitHub Pull Request #5385