Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-947

isReplicationInProgress() is very heavyweight

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      Suppose a decommission is in progress. The namenode receives a heartbeat from the being-decommissioned node and then invokes isReplicationInProgress() to determine if the decommissioned is complete. This method is very heavyweight and chews up plenty of CPU.

      One option to fix this issue is to keep a counter to indicate how many blocks are pending replication. This could also help in showing a progress-status-indicator to display how far the decommissioning process is complete.

        Attachments

        1. isReplicationInProgress2.patch
          2 kB
          Dhruba Borthakur

          Activity

            People

            • Assignee:
              dhruba Dhruba Borthakur
              Reporter:
              dhruba Dhruba Borthakur
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: