[HDDS-6460] EC: Adjust requested size by EC DataNum in WritableECContainerProvider

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: EC-Branch
• Component/s: SCM
• Labels: pull-request-available
• Assignee: Stephen O'Donnell
• Reporter: Stephen O'Donnell
• Created: 16/Mar/22 10:18
• Updated: 22/Mar/22 12:49
• Resolved: 22/Mar/22 12:49

Description

In HDDS-6384 we decided that the size tracked in SCM of an EC container is the usedBytes in the first replica or any parity replica. In WritableECContainerProvider, we currently check if there is enough space in the container by comparing the container UsedBytes against the blockSize requested. However the blockSize needs to be divided by the EC DataNum, as it will be stripped across blocks in the group. There was a TODO left in the code to address this decision later:

          ContainerInfo containerInfo = getContainerFromPipeline(pipeline);
          // TODO - For EC, what is the block size? If the client says 128MB,
          //        is that 128MB / 6 (for EC-6-3?)
          if (containerInfo == null
              || !containerHasSpace(containerInfo, size)) {
            // This is O(n), which isn't great if there are a lot of pipelines
            // and we keep finding pipelines without enough space.
            existingPipelines.remove(pipeline);
            pipelineManager.closePipeline(pipeline, true);

After the decision in HDDS-6384, we should remove this TODO and divide the requested size by the dataNum to get the approx space needed in the first replica to hold the new block.

Issue Links

• links to: GitHub Pull Request #3203