Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5605

All the replicas incorrectly got marked as corrupt.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.20.0
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      NameNode does not handle reportBadBlocks() properly. As a result, when DataNode reports the corruption (only in the case of block transfer between two datanodes), further attempts to replicate the block end up marking all the replicas as corrupt!

      From the implementation, it looks like NN incorrectly uses the block object used in RPC to queue to neededReplication queue instead of using internal block object.

      will include an actual example in the next comment.

        Attachments

        1. reportBadBlock.patch
          2 kB
          Hairong Kuang
        2. reportBadBlock1.patch
          2 kB
          Hairong Kuang
        3. reportBadBlock2.patch
          2 kB
          Hairong Kuang
        4. testReportBadBlock.patch
          5 kB
          Hairong Kuang

          Activity

            People

            • Assignee:
              hairong Hairong Kuang
              Reporter:
              rangadi Raghu Angadi
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: