Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-947

isReplicationInProgress() is very heavyweight

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.12.0
    • None
    • None

    Description

      Suppose a decommission is in progress. The namenode receives a heartbeat from the being-decommissioned node and then invokes isReplicationInProgress() to determine if the decommissioned is complete. This method is very heavyweight and chews up plenty of CPU.

      One option to fix this issue is to keep a counter to indicate how many blocks are pending replication. This could also help in showing a progress-status-indicator to display how far the decommissioning process is complete.

      Attachments

        1. isReplicationInProgress2.patch
          2 kB
          Dhruba Borthakur

        Activity

          People

            dhruba Dhruba Borthakur
            dhruba Dhruba Borthakur
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: