Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13846

Safe blocks counter is not decremented correctly if the block is striped

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.1.0
    • Fix Version/s: 3.2.0
    • Component/s: hdfs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      In BlockManagerSafeMode class, the "safe blocks" counter is incremented if the number of nodes containing the block equals to the number of data units specified by the erasure coding policy, which looks like this in the code:

      final int safe = storedBlock.isStriped() ?
              ((BlockInfoStriped)storedBlock).getRealDataBlockNum() : safeReplication;
          if (storageNum == safe) {
            this.blockSafe++;
      

      But when it is decremented the code does not check if the block is striped or not, just compares the number of nodes containing the block with 0 (safeReplication - 1) if the block is complete, which is not correct.

      if (storedBlock.isComplete() &&
              blockManager.countNodes(b).liveReplicas() == safeReplication - 1) {
            this.blockSafe--;
            assert blockSafe >= 0;
            checkSafeMode();
          }
      

        Attachments

        1. HDFS-13846.001.patch
          5 kB
          Kitti Nanasi
        2. HDFS-13846.002.patch
          6 kB
          Kitti Nanasi
        3. HDFS-13846.003.patch
          8 kB
          Kitti Nanasi
        4. HDFS-13846.004.patch
          8 kB
          Kitti Nanasi
        5. HDFS-13846.005.patch
          8 kB
          Kitti Nanasi

          Activity

            People

            • Assignee:
              knanasi Kitti Nanasi
              Reporter:
              knanasi Kitti Nanasi
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: