Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3685

Unbalanced replication target

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.17.0
    • 0.17.2
    • None
    • None
    • Reviewed

    Description

      In HADOOP-3633, namenode was assigning some datanodes to receive hundreds of blocks in a short period which caused datanodes to go out of memroy(threads).
      Most of them were from remote rack.

      Looking at the code,

          166           chooseLocalRack(results.get(1), excludedNodes, blocksize,
          167                           maxNodesPerRack, results);
      

      was sometimes not choosing the local rack of the writer(source).

      As a result, when a datanode goes down, other datanodes on the same rack were getting large number of blocks from remote racks.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            hairong Hairong Kuang Assign to me
            knoguchi Koji Noguchi
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment