Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.17.0
    • Fix Version/s: 0.17.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In HADOOP-3633, namenode was assigning some datanodes to receive hundreds of blocks in a short period which caused datanodes to go out of memroy(threads).
      Most of them were from remote rack.

      Looking at the code,

          166           chooseLocalRack(results.get(1), excludedNodes, blocksize,
          167                           maxNodesPerRack, results);
      

      was sometimes not choosing the local rack of the writer(source).

      As a result, when a datanode goes down, other datanodes on the same rack were getting large number of blocks from remote racks.

        Attachments

        1. rereplicationPolicy.patch
          10 kB
          Hairong Kuang
        2. rereplicationPolicy1.patch
          10 kB
          Hairong Kuang

          Issue Links

            Activity

              People

              • Assignee:
                hairong Hairong Kuang
                Reporter:
                knoguchi Koji Noguchi
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: