[HDDS-6384] EC: Ensure EC container usage is updated correctly when handling reports

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: EC-Branch
• Component/s: SCM
• Labels: pull-request-available
• Assignee: Stephen O'Donnell
• Reporter: Stephen O'Donnell
• Created: 28/Feb/22 15:54
• Updated: 04/Mar/22 19:31
• Resolved: 04/Mar/22 19:31

Description

The current container report handling logic in AbstractContainerReportHandler does update the container usage and key count correctly in SCM for EC containers. The current logic says:

for (ContainerReplica r : otherReplicas) {
  // Open containers are generally growing in key count and size, the
  // overall size should be the min of all reported replicas.
  if (containerInfo.getState().equals(HddsProtos.LifeCycleState.OPEN)) {
    usedBytes = Math.min(usedBytes, r.getBytesUsed());
    keyCount = Math.min(keyCount, r.getKeyCount());
  } else {
    // Containers which are not open can only shrink in size, so use the
    // largest values reported.
    usedBytes = Math.max(usedBytes, r.getBytesUsed());
    keyCount = Math.max(keyCount, r.getKeyCount());
  }
} 

For EC Containers, the bytes used will be different across all the replicas due to partial stripes. The first data replica and the parity replicas should have the max size, but other data replicas are likely to be smaller. Therefore using the min as above will not work.

We should either use the max value for open containers, or the min of the first data and parity group, and ignore the usage in the other replicas.

Issue Links

• links to: GitHub Pull Request #3147