Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10987

Make Decommission less expensive when lot of blocks present.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When user want to decommission a node which having 50M blocks ,it could hold the namesystem lock for long time.We've seen it is taking 36 sec.

      As we knew during this time, Namenode will not available... As this decommission will continuosly run till all the blocks got replicated,hence Namenode will unavailable.

      1. HDFS-10987.patch
        2 kB
        Brahma Reddy Battula
      2. HDFS-10987-002.patch
        3 kB
        Brahma Reddy Battula

        Activity

        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        We can notice from the following, write lock time ~35sec+ . This I seen in 2.7, same issue exists in trunk also.

        2016-09-26 21:09:15,673 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299
        2016-09-26 21:09:45,645 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398

        2016-09-26 21:09:45,647 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299
        2016-09-26 21:10:15,279 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398

        2016-09-26 21:10:15,280 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299
        2016-09-26 21:10:45,595 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398

        2016-09-26 21:10:50,346 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299
        2016-09-26 21:11:20,176 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398

        2016-09-26 21:11:20,179 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299
        2016-09-26 21:11:49,669 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398

        2016-09-26 21:11:49,670 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299
        2016-09-26 21:12:19,192 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398

        Show
        brahmareddy Brahma Reddy Battula added a comment - We can notice from the following, write lock time ~35sec+ . This I seen in 2.7, same issue exists in trunk also. 2016-09-26 21:09:15,673 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299 2016-09-26 21:09:45,645 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398 2016-09-26 21:09:45,647 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299 2016-09-26 21:10:15,279 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398 2016-09-26 21:10:15,280 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299 2016-09-26 21:10:45,595 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398 2016-09-26 21:10:50,346 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299 2016-09-26 21:11:20,176 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398 2016-09-26 21:11:20,179 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299 2016-09-26 21:11:49,669 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398 2016-09-26 21:11:49,670 | INFO | DecommissionMonitor-0 | Block: blk_1075238065_1497241, Expected Replicas: 3, live replicas: 2, corrupt replicas: 0, decommissioned replicas: 0, decommissioning replicas: 1, excess replicas: 0, Is Open File: false, Datanodes having this block: X.X.X.X:25009 Y.Y.Y.Y:25009 D.D.D.D:25009 , Current Datanode: D.D.D.D:25009, Is current datanode decommissioning: true | DecommissionManager.java:299 2016-09-26 21:12:19,192 | INFO | DecommissionMonitor-0 | Checked 50635805 blocks and 1 nodes this tick | DecommissionManager.java:398
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Yielding is required in case of single Node have huge number of blocks to process.Introduced same and uploading the patch.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Yielding is required in case of single Node have huge number of blocks to process.Introduced same and uploading the patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 6m 50s trunk passed
        +1 compile 0m 43s trunk passed
        +1 checkstyle 0m 24s trunk passed
        +1 mvnsite 0m 50s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 41s trunk passed
        +1 javadoc 0m 39s trunk passed
        +1 mvninstall 0m 44s the patch passed
        +1 compile 0m 40s the patch passed
        +1 javac 0m 40s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 45s the patch passed
        +1 javadoc 0m 37s the patch passed
        -1 unit 58m 3s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        76m 7s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus
          hadoop.hdfs.TestEncryptionZones



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10987
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832259/HDFS-10987.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e6e42fda0aff 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4d10621
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17068/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17068/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17068/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 50s trunk passed +1 compile 0m 43s trunk passed +1 checkstyle 0m 24s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 39s trunk passed +1 mvninstall 0m 44s the patch passed +1 compile 0m 40s the patch passed +1 javac 0m 40s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 58m 3s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 76m 7s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.TestEncryptionZones Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10987 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832259/HDFS-10987.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e6e42fda0aff 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4d10621 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17068/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17068/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17068/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Test failures are unrelated..Kindly review.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Test failures are unrelated..Kindly review.
        Hide
        kihwal Kihwal Lee added a comment -

        We've seen this also. We don't have that many blocks per node, but still the lock time can be multiple seconds. Daryn Sharp was going to do something similar, but he was also improving locking in replication monitor.

        Show
        kihwal Kihwal Lee added a comment - We've seen this also. We don't have that many blocks per node, but still the lock time can be multiple seconds. Daryn Sharp was going to do something similar, but he was also improving locking in replication monitor.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Thanks Kihwal Lee for taking look..

        IIUC,Daryn Sharp was doing finegrained locking which is big change..? what's your view on current patch..?

        JFYI,Tested the patch,NN was available(only first time it will take 15 to 25 sec) while running the decommission and all the depended services(HBase,Spark,Hive..) are able to communicate.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Thanks Kihwal Lee for taking look.. IIUC, Daryn Sharp was doing finegrained locking which is big change..? what's your view on current patch..? JFYI,Tested the patch,NN was available(only first time it will take 15 to 25 sec) while running the decommission and all the depended services(HBase,Spark,Hive..) are able to communicate.
        Hide
        kihwal Kihwal Lee added a comment -

        IIUC,Daryn Sharp was doing finegrained locking which is big change..?

        It is a decomm manager and replication monitor change. He has a POF patch, but got preempted by other things before filing a jira for it.

        Now the patch. It looks okay in general. Just a few minor things.

        // Yielding is required in case of single Node have huge number of
        // blocks to process.
        

        To be precise, the number of blocks doesn't have to be huge. It will yield if the number is greater than the configured per-iteration-limit.
        When the sleep is interrupted, it should probably not ignore. It looks like it can simply return.

        Show
        kihwal Kihwal Lee added a comment - IIUC,Daryn Sharp was doing finegrained locking which is big change..? It is a decomm manager and replication monitor change. He has a POF patch, but got preempted by other things before filing a jira for it. Now the patch. It looks okay in general. Just a few minor things. // Yielding is required in case of single Node have huge number of // blocks to process. To be precise, the number of blocks doesn't have to be huge. It will yield if the number is greater than the configured per-iteration-limit. When the sleep is interrupted, it should probably not ignore. It looks like it can simply return.
        Hide
        andrew.wang Andrew Wang added a comment -

        Nice idea, patch LGTM too once Kihwal's comments are addressed. Hoping we can eventually find a way to break up all the full scans.

        Show
        andrew.wang Andrew Wang added a comment - Nice idea, patch LGTM too once Kihwal's comments are addressed. Hoping we can eventually find a way to break up all the full scans.
        Hide
        vinayrpet Vinayakumar B added a comment -

        To be precise, the number of blocks doesn't have to be huge. It will yield if the number is greater than the configured per-iteration-limit.

        Yes, that's correct. But before-this patch, check against per-iteration-limit is done after checking all blocks-per-node. So yielding is done only after current-nodes list is complete.

        When the sleep is interrupted, it should probably not ignore. It looks like it can simply return.

        Yes. Along with that, IMO should also add 'namesystem.isRunning()' to while loop condition in 'check()' to end execution fast.

        Show
        vinayrpet Vinayakumar B added a comment - To be precise, the number of blocks doesn't have to be huge. It will yield if the number is greater than the configured per-iteration-limit. Yes, that's correct. But before-this patch, check against per-iteration-limit is done after checking all blocks-per-node. So yielding is done only after current-nodes list is complete. When the sleep is interrupted, it should probably not ignore. It looks like it can simply return. Yes. Along with that, IMO should also add 'namesystem.isRunning()' to while loop condition in 'check()' to end execution fast.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Another improvement for standby-nn I am finding is, "blockManager.isPopulatingReplQueues()" could be added at run() itself to avoid all remaining checks in Standby state. Could be taken as separate Jira also.

        Show
        vinayrpet Vinayakumar B added a comment - Another improvement for standby-nn I am finding is, "blockManager.isPopulatingReplQueues()" could be added at run() itself to avoid all remaining checks in Standby state. Could be taken as separate Jira also.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Yes. Along with that, IMO should also add 'namesystem.isRunning()' to while loop condition in 'check()' to end execution fast

        Sounds good to me.. Updated the patch..Addressed kihwal and vinay comments..

        Show
        brahmareddy Brahma Reddy Battula added a comment - Yes. Along with that, IMO should also add 'namesystem.isRunning()' to while loop condition in 'check()' to end execution fast Sounds good to me.. Updated the patch..Addressed kihwal and vinay comments..
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        will handle in seperate jira.

        Show
        brahmareddy Brahma Reddy Battula added a comment - will handle in seperate jira.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 8m 15s trunk passed
        +1 compile 0m 47s trunk passed
        +1 checkstyle 0m 26s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 51s trunk passed
        +1 javadoc 0m 42s trunk passed
        +1 mvninstall 0m 47s the patch passed
        +1 compile 0m 55s the patch passed
        +1 javac 0m 55s the patch passed
        +1 checkstyle 0m 24s the patch passed
        +1 mvnsite 0m 47s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 37s the patch passed
        +1 unit 58m 13s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        78m 31s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10987
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832892/HDFS-10987-002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c874a437d553 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6476934
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17120/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17120/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 15s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 51s trunk passed +1 javadoc 0m 42s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 55s the patch passed +1 javac 0m 55s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 37s the patch passed +1 unit 58m 13s hadoop-hdfs in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 78m 31s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10987 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832892/HDFS-10987-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c874a437d553 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6476934 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17120/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17120/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Looks good to me. +1

        Show
        vinayrpet Vinayakumar B added a comment - Looks good to me. +1
        Hide
        kihwal Kihwal Lee added a comment -

        +1 from me too. Will commit it shortly.

        Show
        kihwal Kihwal Lee added a comment - +1 from me too. Will commit it shortly.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Vinayakumar B thanks for review, will wait for commit till Kihwal Lee and Andrew Wang looks latest patch.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Vinayakumar B thanks for review, will wait for commit till Kihwal Lee and Andrew Wang looks latest patch.
        Hide
        kihwal Kihwal Lee added a comment -

        Ok. Not committing. Let's wait for Andrew.

        Show
        kihwal Kihwal Lee added a comment - Ok. Not committing. Let's wait for Andrew.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        hmm..it was coincidence( same time we commented)..

        Show
        brahmareddy Brahma Reddy Battula added a comment - hmm..it was coincidence( same time we commented)..
        Hide
        andrew.wang Andrew Wang added a comment -

        Yea LGTM too, thanks all!

        Show
        andrew.wang Andrew Wang added a comment - Yea LGTM too, thanks all!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10606 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10606/)
        HDFS-10987. Make Decommission less expensive when lot of blocks present. (kihwal: rev 332a61fd74fd2a9874319232c583ab5d2c53ff03)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10606 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10606/ ) HDFS-10987 . Make Decommission less expensive when lot of blocks present. (kihwal: rev 332a61fd74fd2a9874319232c583ab5d2c53ff03) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        Hide
        kihwal Kihwal Lee added a comment -

        Thanks everyone for reviews and thanks for working on this Brahma Reddy Battula. Committed this to trunk, branch-2 and branch-2.8. branch-2* had a couple of minor conflicts that was easily resolved.

        Show
        kihwal Kihwal Lee added a comment - Thanks everyone for reviews and thanks for working on this Brahma Reddy Battula . Committed this to trunk, branch-2 and branch-2.8. branch-2* had a couple of minor conflicts that was easily resolved.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Thanks all.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Thanks all.

          People

          • Assignee:
            brahmareddy Brahma Reddy Battula
            Reporter:
            brahmareddy Brahma Reddy Battula
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development