Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5479

NameNode should not send empty block replication request to DataNode

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.19.2, 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      On our production clusters, we occasionally see that NameNode sends an empty block replication request to DataNode on every heartbeat, thus blocking this DataNode from replicating or deleting any block.

      This is partly caused by DataNode sending a wrong number of replications in progress which will be fixed by HADOOP-5465. There is also a flaw at the NameNode side. NameNode should not interpret the number of replications in progress as the number of targets since replication is done through a pipeline. It also should make sure that no empty replication request is sent to DataNode.

        Attachments

        1. numTransfers.patch
          3 kB
          Hairong Kuang
        2. numTransfers1.patch
          3 kB
          Hairong Kuang

          Issue Links

            Activity

              People

              • Assignee:
                hairong Hairong Kuang
                Reporter:
                hairong Hairong Kuang
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: