Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-947

isReplicationInProgress() is very heavyweight

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.12.0
    • None
    • None

    Description

      Suppose a decommission is in progress. The namenode receives a heartbeat from the being-decommissioned node and then invokes isReplicationInProgress() to determine if the decommissioned is complete. This method is very heavyweight and chews up plenty of CPU.

      One option to fix this issue is to keep a counter to indicate how many blocks are pending replication. This could also help in showing a progress-status-indicator to display how far the decommissioning process is complete.

      Attachments

        1. isReplicationInProgress2.patch
          2 kB
          Dhruba Borthakur

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            dhruba Dhruba Borthakur
            dhruba Dhruba Borthakur
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment