Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13846

Safe blocks counter is not decremented correctly if the block is striped

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.0
    • 3.2.0
    • hdfs
    • None
    • Reviewed

    Description

      In BlockManagerSafeMode class, the "safe blocks" counter is incremented if the number of nodes containing the block equals to the number of data units specified by the erasure coding policy, which looks like this in the code:

      final int safe = storedBlock.isStriped() ?
              ((BlockInfoStriped)storedBlock).getRealDataBlockNum() : safeReplication;
          if (storageNum == safe) {
            this.blockSafe++;
      

      But when it is decremented the code does not check if the block is striped or not, just compares the number of nodes containing the block with 0 (safeReplication - 1) if the block is complete, which is not correct.

      if (storedBlock.isComplete() &&
              blockManager.countNodes(b).liveReplicas() == safeReplication - 1) {
            this.blockSafe--;
            assert blockSafe >= 0;
            checkSafeMode();
          }
      

      Attachments

        1. HDFS-13846.001.patch
          5 kB
          Kitti Nanasi
        2. HDFS-13846.002.patch
          6 kB
          Kitti Nanasi
        3. HDFS-13846.003.patch
          8 kB
          Kitti Nanasi
        4. HDFS-13846.004.patch
          8 kB
          Kitti Nanasi
        5. HDFS-13846.005.patch
          8 kB
          Kitti Nanasi

        Activity

          People

            knanasi Kitti Nanasi
            knanasi Kitti Nanasi
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: