Uploaded image for project: 'Hadoop Distributed Data Store'
  1. Hadoop Distributed Data Store
  2. HDDS-1559

Include committedBytes to determine Out of Space in VolumeChoosingPolicy

    XMLWordPrintableJSON

    Details

    • Sprint:
      HDDS Biscayne

      Description

      This is a follow-up from HDDS-1511 and HDDS-1535

      Currently when creating a new Container, the DN invokes RoundRobinVolumeChoosingPolicy:chooseVolume(). This routine checks for (volume available space > container max size). If no eligible volume is found, the policy throws a DiskOutOfSpaceException. This is the current behaviour.

      However, the computation of available space does not take into consideration the space
      that is going to be consumed by writes to existing containers which are still Open and accepting chunk writes.

      This Jira proposes to enhance the space availability check in chooseVolume by inclusion of committed space(committedBytes in HddsVolume) in the equation.

      The handling/management of the exception in Ratis will not be modified in this Jira. That will be scoped separately as part of Datanode IO Failure handling work.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sdeka Supratim Deka
                Reporter:
                sdeka Supratim Deka
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 50m
                  1h 50m