Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7116

Add a command to get the balancer bandwidth

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: balancer & mover
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Exposed command "-getBalancerBandwidth" in dfsadmin to get the bandwidth of balancer.

      Description

      Now reading logs is the only way to check how the balancer bandwidth is set. It would be useful for administrators if they can get the value of the same. This jira to discuss & implement the way to access the balancer bandwidth value of the datanode.

      1. HDFS-7116-00.patch
        37 kB
        Rakesh R
      2. HDFS-7116-01.patch
        45 kB
        Rakesh R
      3. HDFS-7116-02.patch
        7 kB
        Rakesh R
      4. HDFS-7116-03.patch
        7 kB
        Rakesh R
      5. HDFS-7116-04.patch
        9 kB
        Rakesh R
      6. HDFS-7116-05.patch
        9 kB
        Rakesh R
      7. HDFS-7116-06.patch
        19 kB
        Rakesh R
      8. HDFS-7116-07.patch
        19 kB
        Rakesh R
      9. HDFS-7116-08.patch
        19 kB
        Rakesh R
      10. HDFS-7116-09.patch
        19 kB
        Rakesh R
      11. HDFS-7116-10.patch
        20 kB
        Rakesh R

        Activity

        Hide
        rakeshr Rakesh R added a comment -

        Hi Akira Ajisaka, its really an interesting feature and would like to take this ahead. To begin with, I have gone through the -setBalancerBandwidth feature. Here, Namenode is sending the DNA_BALANCERBANDWIDTHUPDATE command to the Datanodes as a heartbeat response and Datanode will update the new bandwidth value. In normal case, after setting the bandwidth all the datanodes will have the same value. But I could see few corner cases where all the Datanodes in a cluster not having unique bandwidth value.

        Case-1) Newly added Datanodes will not aware about the new value and will have the default bandwidth value.
        Case-2) Assume we have 10 Datanodes, after sending heartbeat responses to 1 to 5 Datanodes, unfortunately Namenode restarted. Since bandwidth value is not persisted restarted Namenode will not have the new bandwidth value. Now, the rest of the 6 to 10 Datanodes will be having old value.

        Is this expected behavior?. If yes, we have to design the -getBalancerBandwidth command in such a way to list each Datanode with the corresponding bandwidth value like, each Datanode = bandwidth value. Any thoughts?

        Show
        rakeshr Rakesh R added a comment - Hi Akira Ajisaka , its really an interesting feature and would like to take this ahead. To begin with, I have gone through the -setBalancerBandwidth feature. Here, Namenode is sending the DNA_BALANCERBANDWIDTHUPDATE command to the Datanodes as a heartbeat response and Datanode will update the new bandwidth value. In normal case, after setting the bandwidth all the datanodes will have the same value. But I could see few corner cases where all the Datanodes in a cluster not having unique bandwidth value. Case-1) Newly added Datanodes will not aware about the new value and will have the default bandwidth value. Case-2) Assume we have 10 Datanodes, after sending heartbeat responses to 1 to 5 Datanodes, unfortunately Namenode restarted. Since bandwidth value is not persisted restarted Namenode will not have the new bandwidth value. Now, the rest of the 6 to 10 Datanodes will be having old value. Is this expected behavior?. If yes, we have to design the -getBalancerBandwidth command in such a way to list each Datanode with the corresponding bandwidth value like, each Datanode = bandwidth value . Any thoughts?
        Hide
        sekikn Kengo Seki added a comment -

        These are expected behavior, so displaying bandwidth values for each DataNodes seems appropriate. In addition, it may be useful that the command can take a list of DataNodes to be processed, in case the cluster is large.

        Show
        sekikn Kengo Seki added a comment - These are expected behavior, so displaying bandwidth values for each DataNodes seems appropriate. In addition, it may be useful that the command can take a list of DataNodes to be processed, in case the cluster is large.
        Hide
        aw Allen Wittenauer added a comment -

        Why not just add this to dfsadmin -report?

        Show
        aw Allen Wittenauer added a comment - Why not just add this to dfsadmin -report?
        Hide
        rakeshr Rakesh R added a comment -

        Thanks Kengo Seki, Allen Wittenauer. Yes, I also feel dfsadmin -report can show the bandwidth value. Presently 'balancerBandwidth' is kept inside DatanodeDescriptor, am planning to keep the bandwidth value in DatanodeInfo so that datanode report can make use of this value.

        Show
        rakeshr Rakesh R added a comment - Thanks Kengo Seki , Allen Wittenauer . Yes, I also feel dfsadmin -report can show the bandwidth value. Presently 'balancerBandwidth' is kept inside DatanodeDescriptor , am planning to keep the bandwidth value in DatanodeInfo so that datanode report can make use of this value.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks all.

        But I could see few corner cases where all the Datanodes in a cluster not having unique bandwidth value.

        Thanks for the clarification. That's why I need such feature.

        Why not just add this to dfsadmin -report?

        Agree. Adding this information to the command seems fine.

        Show
        ajisakaa Akira Ajisaka added a comment - Thanks all. But I could see few corner cases where all the Datanodes in a cluster not having unique bandwidth value. Thanks for the clarification. That's why I need such feature. Why not just add this to dfsadmin -report? Agree. Adding this information to the command seems fine.
        Hide
        rakeshr Rakesh R added a comment -

        Thanks Akira Ajisaka. I will start preparing a patch by adding balancerBandwidth field in DatanodeInfo.

        Show
        rakeshr Rakesh R added a comment - Thanks Akira Ajisaka . I will start preparing a patch by adding balancerBandwidth field in DatanodeInfo.
        Hide
        rakeshr Rakesh R added a comment -

        Initial design thoughts:

        Case-1) Datanode will inform Namenode about the dfs.datanode.balance.bandwidthPerSec configured value through the heartbeat. Initially I thought of updating Namenode only once during the DatanodeRegistration process, but on a second thought I decided to send it through heartbeat for consistency. On receiving the heartbeat, Namenode HeartbeatManager will update the bandwidth value in {{DatanodeInfo#bandwidth }} field. Then continue to the existing logic of checking a new bandwidth value from admin. Here, this check will be like,

        if ( DatanodeInfo#newBalancerBandwidth > 0 && DatanodeInfo#newBalancerBandwidth != DatanodeInfo#bandwidth )
        

        then sent UPDATE_BANDWIDTH command to Datanode.

        Case-2) Administrator sends -setBalancerBandwidth command. Now, this information will be keeping in a new field newBalancerBandwidth in the DatanodeInfo object.

        Could you please review the algorithm and appreciate review comments. Thanks!

        Show
        rakeshr Rakesh R added a comment - Initial design thoughts: Case-1) Datanode will inform Namenode about the dfs.datanode.balance.bandwidthPerSec configured value through the heartbeat. Initially I thought of updating Namenode only once during the DatanodeRegistration process, but on a second thought I decided to send it through heartbeat for consistency. On receiving the heartbeat, Namenode HeartbeatManager will update the bandwidth value in {{DatanodeInfo#bandwidth }} field. Then continue to the existing logic of checking a new bandwidth value from admin. Here, this check will be like, if ( DatanodeInfo#newBalancerBandwidth > 0 && DatanodeInfo#newBalancerBandwidth != DatanodeInfo#bandwidth ) then sent UPDATE_BANDWIDTH command to Datanode. Case-2) Administrator sends -setBalancerBandwidth command. Now, this information will be keeping in a new field newBalancerBandwidth in the DatanodeInfo object. Could you please review the algorithm and appreciate review comments. Thanks!
        Hide
        ajisakaa Akira Ajisaka added a comment -

        case2 is good to me because case1 adds extra information in heartbeat and extra load for NameNode.

        Show
        ajisakaa Akira Ajisaka added a comment - case2 is good to me because case1 adds extra information in heartbeat and extra load for NameNode.
        Hide
        rakeshr Rakesh R added a comment -

        Thanks! In case-1, we should have a mechanism to send the dfs.datanode.balance.bandwidthPerSec configured value in the Datanode to Namenode. Otherwise Namenode would not be aware about the default value sets at the Datanode, isn't it?

        Show
        rakeshr Rakesh R added a comment - Thanks! In case-1, we should have a mechanism to send the dfs.datanode.balance.bandwidthPerSec configured value in the Datanode to Namenode. Otherwise Namenode would not be aware about the default value sets at the Datanode, isn't it?
        Hide
        ajisakaa Akira Ajisaka added a comment -

        In case-1, we should have a mechanism to send the dfs.datanode.balance.bandwidthPerSec configured value in the Datanode to Namenode. Otherwise Namenode would not be aware about the default value sets at the Datanode, isn't it?

        Agree. If case-2, NameNode cannot get the initial value in the DataNodes. Now I'm +1 for case-1. Thanks Rakesh R.

        Show
        ajisakaa Akira Ajisaka added a comment - In case-1, we should have a mechanism to send the dfs.datanode.balance.bandwidthPerSec configured value in the Datanode to Namenode. Otherwise Namenode would not be aware about the default value sets at the Datanode, isn't it? Agree. If case-2, NameNode cannot get the initial value in the DataNodes. Now I'm +1 for case-1. Thanks Rakesh R .
        Hide
        rakeshr Rakesh R added a comment -

        Thanks again! I hope you are agreeing to send the bandwidth value in every DN heartbeat to NN.

        Show
        rakeshr Rakesh R added a comment - Thanks again! I hope you are agreeing to send the bandwidth value in every DN heartbeat to NN.
        Hide
        rakeshr Rakesh R added a comment -

        I've attached a draft patch to get the initial feedback. The patch is based on the heartbeat approach - Datanode will be sending the bandwidth value in heartbeat message.

        Show
        rakeshr Rakesh R added a comment - I've attached a draft patch to get the initial feedback. The patch is based on the heartbeat approach - Datanode will be sending the bandwidth value in heartbeat message.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 36s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 12 new or modified test files.
        +1 javac 7m 27s There were no new javac warning messages.
        +1 javadoc 9m 36s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 53s The applied patch generated 5 new checkstyle issues (total was 84, now 87).
        -1 whitespace 0m 4s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 35s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 3m 45s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 14s Pre-build of native portion
        -1 hdfs tests 162m 19s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 16s Tests passed in hadoop-hdfs-client.
            205m 44s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.datanode.TestBPOfferService



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12734772/HDFS-7116-00.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / ab7958f
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11101/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11101/artifact/patchprocess/whitespace.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11101/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11101/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11101/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11101/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 12 new or modified test files. +1 javac 7m 27s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 53s The applied patch generated 5 new checkstyle issues (total was 84, now 87). -1 whitespace 0m 4s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 45s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 14s Pre-build of native portion -1 hdfs tests 162m 19s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 16s Tests passed in hadoop-hdfs-client.     205m 44s   Reason Tests Failed unit tests hadoop.hdfs.server.datanode.TestBPOfferService Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12734772/HDFS-7116-00.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / ab7958f checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11101/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11101/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11101/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11101/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11101/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11101/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 16s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 12 new or modified test files.
        +1 javac 7m 26s There were no new javac warning messages.
        +1 javadoc 9m 32s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 54s The applied patch generated 2 new checkstyle issues (total was 84, now 76).
        +1 whitespace 0m 12s The patch has no lines that end in whitespace.
        +1 install 1m 38s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 4m 4s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 12s Pre-build of native portion
        -1 hdfs tests 161m 49s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 16s Tests passed in hadoop-hdfs-client.
            209m 20s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.TestClusterId



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12738551/HDFS-7116-01.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 8d0ef31
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11289/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11289/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11289/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11289/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11289/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 16s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 12 new or modified test files. +1 javac 7m 26s There were no new javac warning messages. +1 javadoc 9m 32s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 54s The applied patch generated 2 new checkstyle issues (total was 84, now 76). +1 whitespace 0m 12s The patch has no lines that end in whitespace. +1 install 1m 38s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 4s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 12s Pre-build of native portion -1 hdfs tests 161m 49s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 16s Tests passed in hadoop-hdfs-client.     209m 20s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestClusterId Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12738551/HDFS-7116-01.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8d0ef31 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11289/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11289/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11289/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11289/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11289/console This message was automatically generated.
        Hide
        rakeshr Rakesh R added a comment -

        1) Test failure is not related to this patch. I've raised separate jira HDFS-8568 to handle the test failure.

        2) It looks like the checkstyle warning was already there in the existing code before the patch.

        ./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java:131:10: More than 7 parameters (found 13).
        ./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java:145:10: More than 7 parameters (found 19).
        

        Akira Ajisaka Could you please review the proposed changes. Thanks!

        Show
        rakeshr Rakesh R added a comment - 1) Test failure is not related to this patch. I've raised separate jira HDFS-8568 to handle the test failure. 2) It looks like the checkstyle warning was already there in the existing code before the patch. ./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java:131:10: More than 7 parameters (found 13). ./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java:145:10: More than 7 parameters (found 19). Akira Ajisaka Could you please review the proposed changes. Thanks!
        Hide
        rakeshr Rakesh R added a comment -

        Hi All,

        Show
        rakeshr Rakesh R added a comment - Hi All,
        Hide
        rakeshr Rakesh R added a comment -

        Hi All,

        In the proposed patch, Datanode is sending balancerBandwidth value to the Namenode through heartbeats. As we know this is done for consistency discussed earlier in this jira. On a second look, I have another idea which will have less overhead.

        • We already have a set of datanode metrics exposed which can be used by admins/monitoring tools. How about exposing balancerBandwidth value as a Datanode metric? Here, admin/monitoring tool has to individually collect the metrics from every Datanode.
        Show
        rakeshr Rakesh R added a comment - Hi All, In the proposed patch, Datanode is sending balancerBandwidth value to the Namenode through heartbeats. As we know this is done for consistency discussed earlier in this jira. On a second look, I have another idea which will have less overhead. We already have a set of datanode metrics exposed which can be used by admins/monitoring tools. How about exposing balancerBandwidth value as a Datanode metric? Here, admin/monitoring tool has to individually collect the metrics from every Datanode.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        How about exposing balancerBandwidth value as a Datanode metric?

        I'm fine with your suggestion. Let's start this first.

        Show
        ajisakaa Akira Ajisaka added a comment - How about exposing balancerBandwidth value as a Datanode metric? I'm fine with your suggestion. Let's start this first.
        Hide
        rakeshr Rakesh R added a comment -

        I'm fine with your suggestion. Let's start this first.

        Thanks a lot Akira Ajisaka for the feedback. I've attached another patch in this jira, here I exposed a new metric named BalancerBandwidth on DataNode. Now, with this approach I'm thinking jira subject/description and the proposed idea is not matching. Do we still have any scope of supporting BalancerBandwidth command. If yes, I'm happy to create a separate jira for the metric changes. Whats your opinion?

        Show
        rakeshr Rakesh R added a comment - I'm fine with your suggestion. Let's start this first. Thanks a lot Akira Ajisaka for the feedback. I've attached another patch in this jira, here I exposed a new metric named BalancerBandwidth on DataNode. Now, with this approach I'm thinking jira subject/description and the proposed idea is not matching. Do we still have any scope of supporting BalancerBandwidth command. If yes, I'm happy to create a separate jira for the metric changes. Whats your opinion?
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Do we still have any scope of supporting BalancerBandwidth command?

        No. If I could get the bandwidth for each DataNode by metrics, I don't need the command. I'll appreciate you will update the subject and description.

        Show
        ajisakaa Akira Ajisaka added a comment - Do we still have any scope of supporting BalancerBandwidth command? No. If I could get the bandwidth for each DataNode by metrics, I don't need the command. I'll appreciate you will update the subject and description.
        Hide
        rakeshr Rakesh R added a comment -

        OK, I've updated the jira subject/desc/release_note accordingly. Please review the proposed patch when you get a chance, Thanks!

        Show
        rakeshr Rakesh R added a comment - OK, I've updated the jira subject/desc/release_note accordingly. Please review the proposed patch when you get a chance, Thanks!
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks Rakesh R for creating the patch.

          private AtomicLong balancerBandwidth = new AtomicLong(0);
        

        There is a class MutableGaugeLong, which is similar to MutableCounterLong and can set arbitrary long value. Would you use the class instead of AtomicLong? The rest of the patch looks good to me.

        Show
        ajisakaa Akira Ajisaka added a comment - Thanks Rakesh R for creating the patch. private AtomicLong balancerBandwidth = new AtomicLong(0); There is a class MutableGaugeLong , which is similar to MutableCounterLong and can set arbitrary long value. Would you use the class instead of AtomicLong ? The rest of the patch looks good to me.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Thanks Rakesh R,
        Patch looks great for adding the metric. This is the simple way to expose the bandwidth value.
        +1 for the patch.

        Can someone confirm whether adding a new metric is a compatible change? I am asking because, a new entry will show up in jmx json.
        If incompatible, need to mark this jira as incompatible.

        Show
        vinayrpet Vinayakumar B added a comment - Thanks Rakesh R , Patch looks great for adding the metric. This is the simple way to expose the bandwidth value. +1 for the patch. Can someone confirm whether adding a new metric is a compatible change? I am asking because, a new entry will show up in jmx json. If incompatible, need to mark this jira as incompatible.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Can someone confirm whether adding a new metric is a compatible change?

        Adding a new metric is compatible. http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/Compatibility.html#MetricsJMX

        Show
        ajisakaa Akira Ajisaka added a comment - Can someone confirm whether adding a new metric is a compatible change? Adding a new metric is compatible. http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/Compatibility.html#MetricsJMX
        Hide
        vinayrpet Vinayakumar B added a comment -

        Thanks @Akira for confirming.

        Show
        vinayrpet Vinayakumar B added a comment - Thanks @Akira for confirming.
        Hide
        rakeshr Rakesh R added a comment -

        Thanks Akira Ajisaka, Vinayakumar B for the comments. Attached another patch addressing the comments.

        Show
        rakeshr Rakesh R added a comment - Thanks Akira Ajisaka , Vinayakumar B for the comments. Attached another patch addressing the comments.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks Rakesh R for updating the patch. I built a pseudo-distributed cluster with your patch and the metric worked well.
        One comment: Would you update the description of "dfsadmin -setBalancerBandwidth" in HDFSCommands document?

        NOTE: The new value is not persistent on the DataNode.
        

        Now the new value is exposed via BalancerBandwidth metric.

        I'm +1 if that is addressed.

        Show
        ajisakaa Akira Ajisaka added a comment - Thanks Rakesh R for updating the patch. I built a pseudo-distributed cluster with your patch and the metric worked well. One comment: Would you update the description of "dfsadmin -setBalancerBandwidth" in HDFSCommands document? NOTE: The new value is not persistent on the DataNode. Now the new value is exposed via BalancerBandwidth metric. I'm +1 if that is addressed.
        Hide
        rakeshr Rakesh R added a comment -

        Attached another patch for the HDFSCommands doc changes. Thanks again!

        Show
        rakeshr Rakesh R added a comment - Attached another patch for the HDFSCommands doc changes. Thanks again!
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks Rakesh R for updating the patch.

        > -| `-setBalancerBandwidth` \<bandwidth in bytes per second\> | Changes the network bandwidth used by each datanode during HDFS block balancing. \<bandwidth\> is the maximum number of bytes per second that will be used by each datanode. This value overrides the dfs.balance.bandwidthPerSec parameter.<C2><A0>NOTE: The new value is not persistent on the DataNode. |
        > +| `-setBalancerBandwidth` \<bandwidth in bytes per second\> | Changes the network bandwidth used by each datanode during HDFS block balancing. \<bandwidth\> is the maximum number of bytes per second that will be used by each datanode. This value overrides the dfs.balance.bandwidthPerSec parameter.<C2><A0>NOTE: The new value is not persistent on the DataNode. The value is exposed via BalancerBandwidth metric. |
        

        1. Would you remove the sentence starts with "The new value is not persistent ... "? If we add the metric, the new value does persist on the DataNodeMetrics.
        2. (minor) Would you replace 0xC2A0 (whitespace in UTF-8) before "NOTE:" with whitespace as well?

        Show
        ajisakaa Akira Ajisaka added a comment - Thanks Rakesh R for updating the patch. > -| `-setBalancerBandwidth` \<bandwidth in bytes per second\> | Changes the network bandwidth used by each datanode during HDFS block balancing. \<bandwidth\> is the maximum number of bytes per second that will be used by each datanode. This value overrides the dfs.balance.bandwidthPerSec parameter.<C2><A0>NOTE: The new value is not persistent on the DataNode. | > +| `-setBalancerBandwidth` \<bandwidth in bytes per second\> | Changes the network bandwidth used by each datanode during HDFS block balancing. \<bandwidth\> is the maximum number of bytes per second that will be used by each datanode. This value overrides the dfs.balance.bandwidthPerSec parameter.<C2><A0>NOTE: The new value is not persistent on the DataNode. The value is exposed via BalancerBandwidth metric. | 1. Would you remove the sentence starts with "The new value is not persistent ... "? If we add the metric, the new value does persist on the DataNodeMetrics. 2. (minor) Would you replace 0xC2A0 (whitespace in UTF-8) before "NOTE:" with whitespace as well?
        Hide
        rakeshr Rakesh R added a comment -

        OK, thank you for the explanation. Attached patch addressing the comments.

        Show
        rakeshr Rakesh R added a comment - OK, thank you for the explanation. Attached patch addressing the comments.
        Hide
        vinayrpet Vinayakumar B added a comment - - edited

        1. Would you remove the sentence starts with "The new value is not persistent ... "? If we add the metric, the new value does persist on the DataNodeMetrics.

        IMO, exposing via metric is not same as persisting. It just shows out the current value. Here NOTE actually meant, new value is only till lifetime of the datanode. On restart it will be reset. so i feel still NOTE is valid. May be the sentence related to exposing can be added before the note. To make it clear.

        Show
        vinayrpet Vinayakumar B added a comment - - edited 1. Would you remove the sentence starts with "The new value is not persistent ... "? If we add the metric, the new value does persist on the DataNodeMetrics. IMO, exposing via metric is not same as persisting. It just shows out the current value. Here NOTE actually meant, new value is only till lifetime of the datanode. On restart it will be reset. so i feel still NOTE is valid. May be the sentence related to exposing can be added before the note. To make it clear.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 23m 35s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 55s There were no new javac warning messages.
        +1 javadoc 9m 55s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 site 2m 59s Site still builds.
        -1 checkstyle 2m 29s The applied patch generated 3 new checkstyle issues (total was 222, now 224).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 27s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 4m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 common tests 22m 41s Tests passed in hadoop-common.
        +1 hdfs tests 163m 6s Tests passed in hadoop-hdfs.
            239m 36s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12751690/HDFS-7116-05.patch
        Optional Tests site javadoc javac unit findbugs checkstyle
        git revision trunk / 22de7c1
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12070/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12070/artifact/patchprocess/testrun_hadoop-common.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12070/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12070/testReport/
        Java 1.7.0_55
        uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12070/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 23m 35s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 55s There were no new javac warning messages. +1 javadoc 9m 55s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 site 2m 59s Site still builds. -1 checkstyle 2m 29s The applied patch generated 3 new checkstyle issues (total was 222, now 224). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 41s Tests passed in hadoop-common. +1 hdfs tests 163m 6s Tests passed in hadoop-hdfs.     239m 36s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12751690/HDFS-7116-05.patch Optional Tests site javadoc javac unit findbugs checkstyle git revision trunk / 22de7c1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12070/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12070/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12070/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12070/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12070/console This message was automatically generated.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        > Expose metric named "BalancerBandwidth" in the Datanode metrics to get the bandwidth of balancer.

        By "bandwidth of balancer", I guess you mean dfs.datanode.balance.bandwidthPerSec, which can be changed in runtime by the "dfsadmin –setBalancerBandwidth" command. This seems wrong to show it in metric since it is a mostly constant value. Just like that we won't show other conf values in metric.

        How about adding a new dfsadmin command?

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - > Expose metric named "BalancerBandwidth" in the Datanode metrics to get the bandwidth of balancer. By "bandwidth of balancer", I guess you mean dfs.datanode.balance.bandwidthPerSec, which can be changed in runtime by the "dfsadmin –setBalancerBandwidth" command. This seems wrong to show it in metric since it is a mostly constant value. Just like that we won't show other conf values in metric. How about adding a new dfsadmin command?
        Hide
        rakeshr Rakesh R added a comment -

        How about adding a new dfsadmin command?

        Thanks Tsz Wo Nicholas Sze for the interest. We have initially discussed to support command or expose the value through datanode report. Could you please see the attached patch 'HDFS-7116-01.patch' , here I tried an attempt by exposing the value through datanode report. Later we've moved to the metric approach due to the heartbeat overhead. Please see the discussion happened earlier in this jira.

        Show
        rakeshr Rakesh R added a comment - How about adding a new dfsadmin command? Thanks Tsz Wo Nicholas Sze for the interest. We have initially discussed to support command or expose the value through datanode report. Could you please see the attached patch ' HDFS-7116 -01.patch' , here I tried an attempt by exposing the value through datanode report. Later we've moved to the metric approach due to the heartbeat overhead. Please see the discussion happened earlier in this jira.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        How about having "dfsadmin -getBalancerBandwidth <DATANODE_HOST:IPC_PORT>", which is similar to "dfsadmin -getDatanodeInfo"?

        Or how about extending the "hdfs getconf" command to support getting conf from datanode?

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - How about having "dfsadmin -getBalancerBandwidth <DATANODE_HOST:IPC_PORT>", which is similar to "dfsadmin -getDatanodeInfo"? Or how about extending the "hdfs getconf" command to support getting conf from datanode?
        Hide
        rakeshr Rakesh R added a comment -

        How about having "dfsadmin -getBalancerBandwidth <DATANODE_HOST:IPC_PORT>", which is similar to "dfsadmin -getDatanodeInfo"?

        Its interesting. Initially I've tried reading values via Namenode and this makes additional overhead to the datanode heartbeat. Now, your approach is avoiding this problem. I'll start preparing a patch if others agree to this approach.

        Show
        rakeshr Rakesh R added a comment - How about having "dfsadmin -getBalancerBandwidth <DATANODE_HOST:IPC_PORT>", which is similar to "dfsadmin -getDatanodeInfo"? Its interesting. Initially I've tried reading values via Namenode and this makes additional overhead to the datanode heartbeat. Now, your approach is avoiding this problem. I'll start preparing a patch if others agree to this approach.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Rakesh R As I discussed with you offline, "dfsadmin -getBalancerBandwidth <DATANODE_HOST:IPC_PORT>" can be better way as nicholas mentioned.

        • No additional overhead and
        • Now User can easily understand get and set method of balancerbandwidth...

        Tsz Wo Nicholas SzeI feel above one should be ok instead of extending the "hdfs getconf" which is currently getting from the configuration files..

        and if everyone agree ,we need to change defect summary also..

        Show
        brahmareddy Brahma Reddy Battula added a comment - Rakesh R As I discussed with you offline, "dfsadmin -getBalancerBandwidth <DATANODE_HOST:IPC_PORT>" can be better way as nicholas mentioned. No additional overhead and Now User can easily understand get and set method of balancerbandwidth... Tsz Wo Nicholas Sze I feel above one should be ok instead of extending the "hdfs getconf" which is currently getting from the configuration files.. and if everyone agree ,we need to change defect summary also..
        Hide
        aw Allen Wittenauer added a comment -

        Could we please stop adding camel case options to command lines?

        Show
        aw Allen Wittenauer added a comment - Could we please stop adding camel case options to command lines?
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        Command options are case insensitive.

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - Command options are case insensitive.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        This seems wrong to show it in metric since it is a mostly constant value. Just like that we won't show other conf values in metric.

        I understand. Thanks Tsz Wo Nicholas Sze for the comment.

        Now, your approach is avoiding this problem. I'll start preparing a patch if others agree to this approach.

        Agree.

        Command options are case insensitive.

        DFSAdmin.java
            try {
              if ("-report".equals(cmd)) {
                report(argv, i);
              } else if ("-safemode".equals(cmd)) {
                setSafeMode(argv, i);
              } else if ("-allowSnapshot".equalsIgnoreCase(cmd)) {
                allowSnapshot(argv);
        ...
        

        dfsadmin command options are case sensitive except allowSnapshot and disallowSnapshot. I'm thinking we should make them case insensitive in a separate jira.

        Show
        ajisakaa Akira Ajisaka added a comment - This seems wrong to show it in metric since it is a mostly constant value. Just like that we won't show other conf values in metric. I understand. Thanks Tsz Wo Nicholas Sze for the comment. Now, your approach is avoiding this problem. I'll start preparing a patch if others agree to this approach. Agree. Command options are case insensitive. DFSAdmin.java try { if ( "-report" .equals(cmd)) { report(argv, i); } else if ( "-safemode" .equals(cmd)) { setSafeMode(argv, i); } else if ( "-allowSnapshot" .equalsIgnoreCase(cmd)) { allowSnapshot(argv); ... dfsadmin command options are case sensitive except allowSnapshot and disallowSnapshot. I'm thinking we should make them case insensitive in a separate jira.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Filed HDFS-8944 for make them case insensitive.

        Show
        ajisakaa Akira Ajisaka added a comment - Filed HDFS-8944 for make them case insensitive.
        Hide
        rakeshr Rakesh R added a comment -

        Thanks everyone for the useful discussions. I've attached patch to support dfsadmin -getBalancerBandwidth <dn_host:ipcport> command, please take a look when you get a chance.

        Show
        rakeshr Rakesh R added a comment - Thanks everyone for the useful discussions. I've attached patch to support dfsadmin -getBalancerBandwidth <dn_host:ipcport> command, please take a look when you get a chance.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 22m 27s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 44s There were no new javac warning messages.
        +1 javadoc 9m 48s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 site 3m 17s Site still builds.
        +1 checkstyle 2m 31s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 4m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 11s Pre-build of native portion
        -1 hdfs tests 61m 21s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 28s Tests passed in hadoop-hdfs-client.
            117m 50s  



        Reason Tests
        Timed out tests org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
          org.apache.hadoop.hdfs.web.TestWebHDFSAcl
          org.apache.hadoop.hdfs.TestPread
          org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12751997/HDFS-7116-06.patch
        Optional Tests javac unit javadoc findbugs checkstyle site
        git revision trunk / feaf034
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12086/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12086/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12086/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12086/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12086/testReport/
        Java 1.7.0_55
        uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12086/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 22m 27s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 44s There were no new javac warning messages. +1 javadoc 9m 48s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 site 3m 17s Site still builds. +1 checkstyle 2m 31s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 11s Pre-build of native portion -1 hdfs tests 61m 21s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 28s Tests passed in hadoop-hdfs-client.     117m 50s   Reason Tests Timed out tests org.apache.hadoop.fs.loadGenerator.TestLoadGenerator   org.apache.hadoop.hdfs.web.TestWebHDFSAcl   org.apache.hadoop.hdfs.TestPread   org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12751997/HDFS-7116-06.patch Optional Tests javac unit javadoc findbugs checkstyle site git revision trunk / feaf034 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12086/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12086/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12086/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12086/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12086/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12086/console This message was automatically generated.
        Hide
        rakeshr Rakesh R added a comment -

        It looks like test case failures are not related to the proposed patch, please ignore it !

        Show
        rakeshr Rakesh R added a comment - It looks like test case failures are not related to the proposed patch, please ignore it !
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Hi Rakesh R, would you rebase the patch for the latest trunk?

        Show
        ajisakaa Akira Ajisaka added a comment - Hi Rakesh R , would you rebase the patch for the latest trunk?
        Hide
        rakeshr Rakesh R added a comment -

        Thanks Akira Ajisaka, I've rebased the patch and attached the same.

        Show
        rakeshr Rakesh R added a comment - Thanks Akira Ajisaka , I've rebased the patch and attached the same.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 23m 17s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 49s There were no new javac warning messages.
        +1 javadoc 9m 54s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        +1 site 2m 58s Site still builds.
        -1 checkstyle 2m 28s The applied patch generated 1 new checkstyle issues (total was 18, now 19).
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 34s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 4m 23s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 5s Pre-build of native portion
        -1 hdfs tests 163m 12s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 27s Tests passed in hadoop-hdfs-client.
            220m 8s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestBalancerBandwidth
          hadoop.hdfs.web.TestWebHDFSOAuth2



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12753697/HDFS-7116-07.patch
        Optional Tests javadoc javac unit findbugs checkstyle site
        git revision trunk / 65ccf2b
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12245/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12245/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12245/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12245/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12245/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 23m 17s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 49s There were no new javac warning messages. +1 javadoc 9m 54s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 site 2m 58s Site still builds. -1 checkstyle 2m 28s The applied patch generated 1 new checkstyle issues (total was 18, now 19). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 23s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 5s Pre-build of native portion -1 hdfs tests 163m 12s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 27s Tests passed in hadoop-hdfs-client.     220m 8s   Reason Tests Failed unit tests hadoop.hdfs.TestBalancerBandwidth   hadoop.hdfs.web.TestWebHDFSOAuth2 Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753697/HDFS-7116-07.patch Optional Tests javadoc javac unit findbugs checkstyle site git revision trunk / 65ccf2b checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12245/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12245/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12245/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12245/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12245/console This message was automatically generated.
        Hide
        rakeshr Rakesh R added a comment -

        Attached another patch fixing test case failures.

        Show
        rakeshr Rakesh R added a comment - Attached another patch fixing test case failures.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 23m 6s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 57s There were no new javac warning messages.
        +1 javadoc 10m 10s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        +1 site 3m 8s Site still builds.
        +1 checkstyle 2m 32s There were no new checkstyle issues.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 41s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 4m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 21s Pre-build of native portion
        -1 hdfs tests 184m 13s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 28s Tests passed in hadoop-hdfs-client.
            242m 9s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestLeaseRecovery2
          hadoop.fs.loadGenerator.TestLoadGenerator
          hadoop.fs.viewfs.TestViewFsFileStatusHdfs
          hadoop.fs.TestFcHdfsCreateMkdir
          hadoop.fs.TestFcHdfsPermission
          hadoop.fs.viewfs.TestViewFsWithXAttrs
          hadoop.fs.TestSWebHdfsFileContextMainOperations
          hadoop.fs.contract.hdfs.TestHDFSContractDelete
          hadoop.fs.TestUrlStreamHandler
          hadoop.fs.TestEnhancedByteBufferAccess
          hadoop.fs.viewfs.TestViewFileSystemWithXAttrs
          hadoop.fs.TestHDFSFileContextMainOperations
          hadoop.fs.viewfs.TestViewFsHdfs
          hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
          hadoop.fs.viewfs.TestViewFileSystemWithAcls
          hadoop.fs.TestSymlinkHdfsFileContext
          hadoop.fs.TestSymlinkHdfsFileSystem
          hadoop.fs.TestUnbuffer
          hadoop.fs.viewfs.TestViewFsWithAcls
          hadoop.fs.TestResolveHdfsSymlink
          hadoop.fs.viewfs.TestViewFsDefaultValue
          hadoop.fs.viewfs.TestViewFileSystemHdfs
          hadoop.fs.viewfs.TestViewFsAtHdfsRoot
          hadoop.fs.permission.TestStickyBit
          hadoop.fs.TestSymlinkHdfsDisable
          hadoop.fs.TestGlobPaths
          hadoop.fs.contract.hdfs.TestHDFSContractSeek
          hadoop.fs.shell.TestHdfsTextCommand
        Timed out tests org.apache.hadoop.fs.contract.hdfs.TestHDFSContractMkdir



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12753753/HDFS-7116-08.patch
        Optional Tests javadoc javac unit findbugs checkstyle site
        git revision trunk / 7d6687f
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12248/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12248/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12248/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12248/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 23m 6s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 57s There were no new javac warning messages. +1 javadoc 10m 10s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 site 3m 8s Site still builds. +1 checkstyle 2m 32s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 41s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 21s Pre-build of native portion -1 hdfs tests 184m 13s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 28s Tests passed in hadoop-hdfs-client.     242m 9s   Reason Tests Failed unit tests hadoop.hdfs.TestLeaseRecovery2   hadoop.fs.loadGenerator.TestLoadGenerator   hadoop.fs.viewfs.TestViewFsFileStatusHdfs   hadoop.fs.TestFcHdfsCreateMkdir   hadoop.fs.TestFcHdfsPermission   hadoop.fs.viewfs.TestViewFsWithXAttrs   hadoop.fs.TestSWebHdfsFileContextMainOperations   hadoop.fs.contract.hdfs.TestHDFSContractDelete   hadoop.fs.TestUrlStreamHandler   hadoop.fs.TestEnhancedByteBufferAccess   hadoop.fs.viewfs.TestViewFileSystemWithXAttrs   hadoop.fs.TestHDFSFileContextMainOperations   hadoop.fs.viewfs.TestViewFsHdfs   hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot   hadoop.fs.viewfs.TestViewFileSystemWithAcls   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.fs.TestSymlinkHdfsFileSystem   hadoop.fs.TestUnbuffer   hadoop.fs.viewfs.TestViewFsWithAcls   hadoop.fs.TestResolveHdfsSymlink   hadoop.fs.viewfs.TestViewFsDefaultValue   hadoop.fs.viewfs.TestViewFileSystemHdfs   hadoop.fs.viewfs.TestViewFsAtHdfsRoot   hadoop.fs.permission.TestStickyBit   hadoop.fs.TestSymlinkHdfsDisable   hadoop.fs.TestGlobPaths   hadoop.fs.contract.hdfs.TestHDFSContractSeek   hadoop.fs.shell.TestHdfsTextCommand Timed out tests org.apache.hadoop.fs.contract.hdfs.TestHDFSContractMkdir Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753753/HDFS-7116-08.patch Optional Tests javadoc javac unit findbugs checkstyle site git revision trunk / 7d6687f hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12248/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12248/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12248/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12248/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 20m 40s Findbugs (version 3.0.0) appears to be broken on trunk.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 45s There were no new javac warning messages.
        +1 javadoc 10m 4s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 site 3m 0s Site still builds.
        +1 checkstyle 2m 10s There were no new checkstyle issues.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 4m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 14s Pre-build of native portion
        +1 hdfs tests 164m 18s Tests passed in hadoop-hdfs.
        +1 hdfs tests 0m 29s Tests passed in hadoop-hdfs-client.
            218m 50s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12753753/HDFS-7116-08.patch
        Optional Tests javadoc javac unit findbugs checkstyle site
        git revision trunk / 09c64ba
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12267/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12267/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12267/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12267/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 20m 40s Findbugs (version 3.0.0) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 45s There were no new javac warning messages. +1 javadoc 10m 4s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 site 3m 0s Site still builds. +1 checkstyle 2m 10s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 14s Pre-build of native portion +1 hdfs tests 164m 18s Tests passed in hadoop-hdfs. +1 hdfs tests 0m 29s Tests passed in hadoop-hdfs-client.     218m 50s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753753/HDFS-7116-08.patch Optional Tests javadoc javac unit findbugs checkstyle site git revision trunk / 09c64ba hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12267/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12267/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12267/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12267/console This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Mostly looks good to me. Minor comments:

           * Usage: hdfs dfsadmin -getBalancerBandwidth &ltdatanode_host:ipc_port&gt
        

        1. &lt is not translated to < because there is no whitespace between &lt and datanode. We can use @literal tag to show < and > instead.

            int exitCode = -1;
        (snip)
            exitCode = 0;
        
            return exitCode;
        

        2. We can just return 0 instead of defining exitCode.

                "\t\t--- NOTE: This value is not persistent on the DataNode.---\n";
        

        3. I'm thinking one \t is sufficient.

              System.out.println("Balancer bandwidth value is " + bandwidth
                  + "bytes per second.");
        

        4. Would you add a whitespace before "bytes per second."?
        5. I'm thinking "value" is redundant. Would you remove it to keep consistent with the message of setBalancerBandwidth command?

        Show
        ajisakaa Akira Ajisaka added a comment - Mostly looks good to me. Minor comments: * Usage: hdfs dfsadmin -getBalancerBandwidth &ltdatanode_host:ipc_port&gt 1. &lt is not translated to < because there is no whitespace between &lt and datanode. We can use @literal tag to show < and > instead. int exitCode = -1; (snip) exitCode = 0; return exitCode; 2. We can just return 0 instead of defining exitCode. "\t\t--- NOTE: This value is not persistent on the DataNode.---\n" ; 3. I'm thinking one \t is sufficient. System .out.println( "Balancer bandwidth value is " + bandwidth + "bytes per second." ); 4. Would you add a whitespace before "bytes per second."? 5. I'm thinking "value" is redundant. Would you remove it to keep consistent with the message of setBalancerBandwidth command?
        Hide
        rakeshr Rakesh R added a comment -

        Thank you Akira Ajisaka for the review comments. Attached another patch addressing the same.

        Following is the -help getBalancerBandwidth message.

        -getBalancerBandwidth <datanode_host:ipc_port>:
        	Get the network bandwidth for the given datanode.
        	This is the maximum network bandwidth used by the datanode
        	during HDFS block balancing.
        
        	--- NOTE: This value is not persistent on the DataNode.---
        
        Show
        rakeshr Rakesh R added a comment - Thank you Akira Ajisaka for the review comments. Attached another patch addressing the same. Following is the -help getBalancerBandwidth message. -getBalancerBandwidth <datanode_host:ipc_port>: Get the network bandwidth for the given datanode. This is the maximum network bandwidth used by the datanode during HDFS block balancing. --- NOTE: This value is not persistent on the DataNode.---
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Following is the -help getBalancerBandwidth message.

        Thank you for updating the patch. I'm +1 pending Jenkins.
        Hi Tsz Wo Nicholas Sze and Vinayakumar B, could you take a look?

        Show
        ajisakaa Akira Ajisaka added a comment - Following is the -help getBalancerBandwidth message. Thank you for updating the patch. I'm +1 pending Jenkins. Hi Tsz Wo Nicholas Sze and Vinayakumar B , could you take a look?
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 22m 15s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 42s There were no new javac warning messages.
        +1 javadoc 9m 55s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 site 2m 59s Site still builds.
        +1 checkstyle 2m 30s There were no new checkstyle issues.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 37s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 4m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 9s Pre-build of native portion
        -1 hdfs tests 189m 53s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 30s Tests passed in hadoop-hdfs-client.
            245m 56s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestRollingUpgrade
          hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.TestBalancerBandwidth
          hadoop.hdfs.security.TestDelegationToken



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12753961/HDFS-7116-09.patch
        Optional Tests javadoc javac unit findbugs checkstyle site
        git revision trunk / c92e31b
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12275/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12275/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12275/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12275/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 22m 15s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 9m 55s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 site 2m 59s Site still builds. +1 checkstyle 2m 30s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 9s Pre-build of native portion -1 hdfs tests 189m 53s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 30s Tests passed in hadoop-hdfs-client.     245m 56s   Reason Tests Failed unit tests hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.TestBalancerBandwidth   hadoop.hdfs.security.TestDelegationToken Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753961/HDFS-7116-09.patch Optional Tests javadoc javac unit findbugs checkstyle site git revision trunk / c92e31b hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12275/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12275/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12275/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12275/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        +1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 22m 56s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 8m 5s There were no new javac warning messages.
        +1 javadoc 10m 21s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 site 3m 4s Site still builds.
        +1 checkstyle 2m 30s There were no new checkstyle issues.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 39s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 4m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 23s Pre-build of native portion
        +1 hdfs tests 163m 32s Tests passed in hadoop-hdfs.
        +1 hdfs tests 0m 29s Tests passed in hadoop-hdfs-client.
            221m 34s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12753961/HDFS-7116-09.patch
        Optional Tests javadoc javac unit findbugs checkstyle site
        git revision trunk / c83d13c
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12300/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12300/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12300/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12300/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 22m 56s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 5s There were no new javac warning messages. +1 javadoc 10m 21s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 site 3m 4s Site still builds. +1 checkstyle 2m 30s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 23s Pre-build of native portion +1 hdfs tests 163m 32s Tests passed in hadoop-hdfs. +1 hdfs tests 0m 29s Tests passed in hadoop-hdfs-client.     221m 34s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753961/HDFS-7116-09.patch Optional Tests javadoc javac unit findbugs checkstyle site git revision trunk / c83d13c hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12300/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12300/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12300/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12300/console This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Thanks Rakesh R for the effort on this.

        Source changes looks great.

        Little test improvement required

         
        +  private void runGetBalancerBandwidthCmd(DFSAdmin admin, String[] args,
        +      ClientDatanodeProtocol proxy, long expectedBandwidth) throws Exception {
        +    int exitCode = admin.run(args);
        +    assertEquals("DFSAdmin should return 0", 0, exitCode);
        +    assertEquals("Failed to get the balancer bandwidth!", expectedBandwidth,
        +        proxy.getBalancerBandwidth());
        +  }
        +

        I think just verifying the exit code is not enough. Expected bandwidth should be verified against the output of the command itself instead of getting it using another dn proxy.

        +1 once addressed

        Show
        vinayrpet Vinayakumar B added a comment - Thanks Rakesh R for the effort on this. Source changes looks great. Little test improvement required + private void runGetBalancerBandwidthCmd(DFSAdmin admin, String [] args, + ClientDatanodeProtocol proxy, long expectedBandwidth) throws Exception { + int exitCode = admin.run(args); + assertEquals( "DFSAdmin should return 0" , 0, exitCode); + assertEquals( "Failed to get the balancer bandwidth!" , expectedBandwidth, + proxy.getBalancerBandwidth()); + } + I think just verifying the exit code is not enough. Expected bandwidth should be verified against the output of the command itself instead of getting it using another dn proxy. +1 once addressed
        Hide
        rakeshr Rakesh R added a comment -

        Thanks a lot Akira Ajisaka and Vinayakumar B for the reviews. Attached another patch addressing above comment, kindly review the changes!

        Show
        rakeshr Rakesh R added a comment - Thanks a lot Akira Ajisaka and Vinayakumar B for the reviews. Attached another patch addressing above comment, kindly review the changes!
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 22m 36s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 54s There were no new javac warning messages.
        +1 javadoc 10m 6s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 site 3m 4s Site still builds.
        +1 checkstyle 2m 28s There were no new checkstyle issues.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 35s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 4m 21s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 10s Pre-build of native portion
        -1 hdfs tests 171m 26s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 38s Tests passed in hadoop-hdfs-client.
            228m 17s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.blockmanagement.TestBlockManager
        Timed out tests org.apache.hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12754893/HDFS-7116-10.patch
        Optional Tests javadoc javac unit findbugs checkstyle site
        git revision trunk / 0113e45
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12352/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12352/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12352/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12352/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 22m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 54s There were no new javac warning messages. +1 javadoc 10m 6s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 site 3m 4s Site still builds. +1 checkstyle 2m 28s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 21s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 10s Pre-build of native portion -1 hdfs tests 171m 26s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 38s Tests passed in hadoop-hdfs-client.     228m 17s   Reason Tests Failed unit tests hadoop.hdfs.server.blockmanagement.TestBlockManager Timed out tests org.apache.hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754893/HDFS-7116-10.patch Optional Tests javadoc javac unit findbugs checkstyle site git revision trunk / 0113e45 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12352/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/12352/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12352/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12352/console This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        I think tests failures are unrelated

        +1 for the latest patch

        Show
        vinayrpet Vinayakumar B added a comment - I think tests failures are unrelated +1 for the latest patch
        Hide
        vinayrpet Vinayakumar B added a comment -

        Committed to trunk and branch-2

        Show
        vinayrpet Vinayakumar B added a comment - Committed to trunk and branch-2
        Hide
        vinayrpet Vinayakumar B added a comment -

        Thanks Rakesh R for the patience and contribution.
        Thanks Akira Ajisaka for reporting.
        Thanks All for reviews and discussions.

        Show
        vinayrpet Vinayakumar B added a comment - Thanks Rakesh R for the patience and contribution. Thanks Akira Ajisaka for reporting. Thanks All for reviews and discussions.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1100 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1100/)
        HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1100 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1100/ ) HDFS-7116 . Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8422 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8422/)
        HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
        • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8422 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8422/ ) HDFS-7116 . Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #362 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/362/)
        HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto
        • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #362 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/362/ ) HDFS-7116 . Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #370 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/370/)
        HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #370 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/370/ ) HDFS-7116 . Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #2288 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2288/)
        HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2288 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2288/ ) HDFS-7116 . Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #350 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/350/)
        HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #350 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/350/ ) HDFS-7116 . Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2312 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2312/)
        HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772)

        • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2312 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2312/ ) HDFS-7116 . Add a command to get the balancer bandwidth (Contributed by Rakesh R) (vinayakumarb: rev 0f0e897bf1ff8383f3f524ba83161969531b0772) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBalancerBandwidth.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientDatanodeProtocol.proto hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        rakeshr Rakesh R added a comment -

        Thanks Vinayakumar B, Akira Ajisaka, Tsz Wo Nicholas Sze, all others for the useful discussions and reviews!

        Show
        rakeshr Rakesh R added a comment - Thanks Vinayakumar B , Akira Ajisaka , Tsz Wo Nicholas Sze , all others for the useful discussions and reviews!

          People

          • Assignee:
            rakeshr Rakesh R
            Reporter:
            ajisakaa Akira Ajisaka
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development