Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-145

FSNameSystem#addStoredBlock does not handle inconsistent block length correctly



    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.21.0
    • 0.21.0
    • None
    • None
    • Reviewed


      Currently NameNode treats either the new replica or existing replicas as corrupt if the new replica's length is inconsistent with NN recorded block length. The correct behavior should be
      1. For a block that is not under construction, the new replica should be marked as corrupt if its length is inconsistent (no matter shorter or longer) with the NN recorded block length;
      2. For an under construction block, if the new replica's length is shorter than the NN recorded block length, the new replica could be marked as corrupt; if the new replica's length is longer, NN should update its recorded block length. But it should not mark existing replicas as corrupt. This is because NN recorded length for an under construction block does not accurately match the block length on datanode disk. NN should not judge an under construction replica to be corrupt by looking at the inaccurate information: its recorded block length.


        1. inconsistentLen.patch
          5 kB
          Hairong Kuang
        2. inconsistentLen1.patch
          8 kB
          Hairong Kuang
        3. inconsistentLen2.patch
          17 kB
          Hairong Kuang
        4. corruptionDetect.patch
          9 kB
          Hairong Kuang
        5. corruptionDetect1.patch
          9 kB
          Hairong Kuang
        6. corruptionDetect2.patch
          9 kB
          Hairong Kuang



            hairong Hairong Kuang
            hairong Hairong Kuang
            0 Vote for this issue
            6 Start watching this issue