Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8863

The remaining space check in BlockPlacementPolicyDefault is flawed

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The block placement policy calls DatanodeDescriptor#getRemaining(StorageType}} to check whether the block is going to fit. Since the method is adding up all remaining spaces, namenode can allocate a new block on a full node. This causes pipeline construction failure and abandonBlock. If the cluster is nearly full, the client might hit this multiple times and the write can fail permanently.

        Attachments

        1. HDFS-8863-branch-2.6.1.txt
          10 kB
          Vinod Kumar Vavilapalli
        2. HDFS-8863.v3.patch
          9 kB
          Kihwal Lee
        3. HDFS-8863.v2.patch
          7 kB
          Kihwal Lee
        4. HDFS-8863.patch
          7 kB
          Kihwal Lee

          Issue Links

            Activity

              People

              • Assignee:
                kihwal Kihwal Lee
                Reporter:
                kihwal Kihwal Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: