[HDDS-8617] Ratis underreplication due to maintenance is not deprioritised

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.4.0
• Component/s: SCM
• Labels: pull-request-available
• Target Version/s: 1.4.0
• Assignee: Attila Doroszlai
• Reporter: Attila Doroszlai
• Created: 12/May/23 20:29
• Updated: 19/May/23 20:39
• Resolved: 19/May/23 20:39

Description

According to the following javadoc, both decommission and maintenance replicas should be deprioritised:

https://github.com/apache/ozone/blob/6d9002201e58dc995dc133941acaef2af03cb9d2/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ContainerHealthResult.java#L145-L164

    /**
     * The weightedRedundancy, is the remaining redundancy + the requeue count.
     * When this value is used for ordering in a priority queue it ensures the
     * priority is reduced each time it is requeued, to prevent it from blocking
     * other containers from being processed.
     * Additionally, so that decommission and maintenance replicas are not
     * ordered ahead of under-replicated replicas, a redundancy of
     * DECOMMISSION_REDUNDANCY is used for the decommission redundancy rather
     * than its real redundancy.
     * @return The weightedRedundancy of this result.
     */
    public int getWeightedRedundancy() {
      int result = requeueCount;
      if (dueToDecommission) {
        result += DECOMMISSION_REDUNDANCY;
      } else {
        result += getRemainingRedundancy();
      }
      return result;
    }

but dueToDecommission=true is set only based on decommission replicas, ignoring maintenance replicas (maintenanceCount):

https://github.com/apache/ozone/blob/6d9002201e58dc995dc133941acaef2af03cb9d2/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/RatisContainerReplicaCount.java#L520-L533

  /**
   * Checks whether insufficient replication is because of some replicas
   * being on datanodes that were decommissioned.
   * @param includePendingAdd if pending adds should be considered
   * @return true if there is insufficient replication and it's because of
   * decommissioning.
   */
  public boolean inSufficientDueToDecommission(boolean includePendingAdd) {
    if (isSufficientlyReplicated(includePendingAdd)) {
      return false;
    }
    int delta = redundancyDelta(true, includePendingAdd);
    return decommissionCount >= delta;
  }

Issue Links

• links to: GitHub Pull Request #4735