Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10843

Update space quota when a UC block is completed rather than committed.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha2
    • Component/s: hdfs, namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently when a block has been committed but has not yet been completed, the cached size (used for the quota feature) of the directory containing that block differs from the computed size. This results in log messages of the following form:

      ERROR namenode.NameNode (DirectoryWithQuotaFeature.java:checkStoragespace(141)) - BUG: Inconsistent storagespace for directory /TestQuotaUpdate. Cached = 512 != Computed = 8192

      When a block is initially started under construction, the used space is conservatively set to a full block. When the block is committed, the cached size is updated to the final size of the block. However, the calculation of the computed size uses the full block size until the block is completed, so in the period where the block is committed but not completed they disagree. To fix this we need to decide which is correct and fix the other to match. It seems to me that the cached size is correct since once the block is committed its size will not change.

      This can be reproduced using the following steps:

      • Create a directory with a quota
      • Start writing to a file within this directory
      • Prevent all datanodes to which the file is written from communicating the corresponding BlockReceivedAndDeletedRequestProto to the NN temporarily (i.e. simulate a transient network partition/delay)
      • During this time, call DistributedFileSystem.getContentSummary() on the directory with the quota

        Attachments

        1. HDFS-10843-branch-2.7.005.patch
          33 kB
          Erik Krogen
        2. HDFS-10843-branch-2.005.patch
          32 kB
          Konstantin Shvachko
        3. HDFS-10843.005.patch
          37 kB
          Erik Krogen
        4. HDFS-10843.004.patch
          38 kB
          Erik Krogen
        5. HDFS-10843.003.patch
          35 kB
          Erik Krogen
        6. HDFS-10843.002.patch
          33 kB
          Erik Krogen
        7. HDFS-10843.001.patch
          14 kB
          Erik Krogen
        8. HDFS-10843.000.patch
          14 kB
          Erik Krogen

          Issue Links

            Activity

              People

              • Assignee:
                xkrogen Erik Krogen
                Reporter:
                xkrogen Erik Krogen
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: