Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2202

Changes to balancer bandwidth should not require datanode restart.

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0, 0.23.0
    • Fix Version/s: 0.20.205.0, 0.23.0
    • Component/s: balancer, datanode
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      New dfsadmin command added: [-setBalancerBandwidth <bandwidth>] where bandwidth is max network bandwidth in bytes per second that the balancer is allowed to use on each datanode during balacing.

      This is an incompatible change in 0.23. The versions of ClientProtocol and DatanodeProtocol are changed.
      Show
      New dfsadmin command added: [-setBalancerBandwidth <bandwidth>] where bandwidth is max network bandwidth in bytes per second that the balancer is allowed to use on each datanode during balacing. This is an incompatible change in 0.23. The versions of ClientProtocol and DatanodeProtocol are changed.

      Description

      Currently in order to change the value of the balancer bandwidth (dfs.datanode.balance.bandwidthPerSec), the datanode daemon must be restarted.

      The optimal value of the bandwidthPerSec parameter is not always (almost never) known at the time of cluster startup, but only once a new node is placed in the cluster and balancing is begun. If the balancing is taking too long (bandwidthPerSec is too low) or the balancing is taking up too much bandwidth (bandwidthPerSec is too high), the cluster must go into a "maintenance window" where it is unusable while all of the datanodes are bounced. In large clusters of thousands of nodes, this can be a real maintenance problem because these "mainenance windows" can take a long time and there may have to be several of them while the bandwidthPerSec is experimented with and tuned.

      A possible solution to this problem would be to add a -bandwidth parameter to the balancer tool. If bandwidth is supplied, pass the value to the datanodes via the OP_REPLACE_BLOCK and OP_COPY_BLOCK DataTransferProtocol requests. This would make it necessary, however, to change the DataTransferProtocol version.

      1. HDFS-2171.patch
        21 kB
        Eric Payne
      2. HDFS-2202.patch
        21 kB
        Eric Payne
      3. HDFS-2202.0.23.0.v1.patch
        24 kB
        Eric Payne
      4. HDFS-2202.0.20.205.0.v1.patch
        24 kB
        Eric Payne
      5. HDFS-2202.0.23.0.v2.patch
        24 kB
        Eric Payne
      6. HDFS-2202.0.20.205.0.v2.patch
        24 kB
        Eric Payne
      7. ant.test.0.23.out
        52 kB
        Eric Payne
      8. ASF.LICENSE.NOT.GRANTED--Balancer Bandwidth MSC.jpg
        52 kB
        Eric Payne

        Issue Links

          Activity

          Eric Payne created issue -
          Eric Payne made changes -
          Field Original Value New Value
          Link This issue is a clone of HDFS-2171 [ HDFS-2171 ]
          Nathan Roberts made changes -
          Fix Version/s 0.23.0 [ 12315571 ]
          Fix Version/s 0.20.205.0 [ 12316392 ]
          Hide
          Eric Payne added a comment -

          See HDFS-2171 for testing information.

          Show
          Eric Payne added a comment - See HDFS-2171 for testing information.
          Eric Payne made changes -
          Attachment HDFS-2171.patch [ 12487905 ]
          Eric Payne made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note New dfsadmin command added: [-setBalancerBandwidth <bandwidth>] where bandwidth is max network bandwidth in bytes per second that the balancer is allowed to use on each datanode during balacing.
          Hide
          Eric Payne added a comment -

          Please ignore previous attachment. It is the same content, but it had the wrong name.

          Show
          Eric Payne added a comment - Please ignore previous attachment. It is the same content, but it had the wrong name.
          Eric Payne made changes -
          Attachment HDFS-2202.patch [ 12487909 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12487905/HDFS-2171.patch
          against trunk revision 1151238.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.cli.TestHDFSCLI
          org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery
          org.apache.hadoop.hdfs.server.namenode.TestBackupNode

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1032//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1032//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1032//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12487905/HDFS-2171.patch against trunk revision 1151238. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery org.apache.hadoop.hdfs.server.namenode.TestBackupNode +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1032//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1032//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1032//console This message is automatically generated.
          Hide
          Eric Payne added a comment -

          1) The core tests failures seem to be unrelated. These DO NOT show up when I run test-patch nore when I run test-core in my build environment. I'm looking into those.
          2) HDFS-2106 has refactored the FSNameSystem class (and others), so this HDFS-2202 patch will also need to be redone.

          Show
          Eric Payne added a comment - 1) The core tests failures seem to be unrelated. These DO NOT show up when I run test-patch nore when I run test-core in my build environment. I'm looking into those. 2) HDFS-2106 has refactored the FSNameSystem class (and others), so this HDFS-2202 patch will also need to be redone.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > 2) HDFS-2106 has refactored the FSNameSystem class (and others), ...

          Hi Eric, sorry that the refactoring breaks your patch. Could you update it?

          • In TestBalancerBandwidth, you may call MiniDFSCluster.getFileSystem() instead of creating a DFSClient.
          • We should update ClientProtocol.versionID and DatanodeProtocol.versionID.
          • I think the BalancerBandwidthCommand.version
          Show
          Tsz Wo Nicholas Sze added a comment - > 2) HDFS-2106 has refactored the FSNameSystem class (and others), ... Hi Eric, sorry that the refactoring breaks your patch. Could you update it? In TestBalancerBandwidth, you may call MiniDFSCluster.getFileSystem() instead of creating a DFSClient. We should update ClientProtocol.versionID and DatanodeProtocol.versionID. I think the BalancerBandwidthCommand.version
          Hide
          Tsz Wo Nicholas Sze added a comment -

          (clicked 'add' accidenticaly)

          • I think the BalancerBandwidthCommand.version is not needed. We have to change the DatanodeProtocol.versionID in this case.
          • You may use for-each statement for the following
            +      for(Iterator<DatanodeDescriptor> it = datanodeMap.values().iterator(); 
            +          it.hasNext();) {
            +        DatanodeDescriptor nodeInfo = it.next();
            +        nodeInfo.bandwidth = bandwidth;
            +      }
            

            i.e.

            for(DatanodeDescriptor nodeInfo : datanodeMap.values()) {
               ...
            }
            
          • The initial capacity does not really matter. How about removing it?
            -        ArrayList<DatanodeCommand> cmds = new ArrayList<DatanodeCommand>(3);
            +        ArrayList<DatanodeCommand> cmds = new ArrayList<DatanodeCommand>(4);
            

            I think it is better to declare cmds as

                    final List<DatanodeCommand> cmds = new ArrayList<DatanodeCommand>();
            
          • Please add getter/setter and do not use public field DatanodeDescriptor.bandwidth.
          • Please add javadoc (or change comments to javadoc) to all new public classes/methods/fields.
          Show
          Tsz Wo Nicholas Sze added a comment - (clicked 'add' accidenticaly) I think the BalancerBandwidthCommand.version is not needed. We have to change the DatanodeProtocol.versionID in this case. You may use for-each statement for the following + for (Iterator<DatanodeDescriptor> it = datanodeMap.values().iterator(); + it.hasNext();) { + DatanodeDescriptor nodeInfo = it.next(); + nodeInfo.bandwidth = bandwidth; + } i.e. for (DatanodeDescriptor nodeInfo : datanodeMap.values()) { ... } The initial capacity does not really matter. How about removing it? - ArrayList<DatanodeCommand> cmds = new ArrayList<DatanodeCommand>(3); + ArrayList<DatanodeCommand> cmds = new ArrayList<DatanodeCommand>(4); I think it is better to declare cmds as final List<DatanodeCommand> cmds = new ArrayList<DatanodeCommand>(); Please add getter/setter and do not use public field DatanodeDescriptor.bandwidth. Please add javadoc (or change comments to javadoc) to all new public classes/methods/fields.
          Hide
          Eric Payne added a comment -

          I am duplicating HDFS-2171 to this Jira (HDFS-2202) so as to make the review and commit steps easier. HDFS-2171 was originally for 0.20.205 and HDFS-2202 was originally for 0.23. Now, HDFS-2202 is for both

          Please see Requirements and Design Tradeoffs here:
          https://issues.apache.org/jira/browse/HDFS-2171?focusedCommentId=13068990&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13068990

          Please see junit and manual test information here:
          https://issues.apache.org/jira/browse/HDFS-2171?focusedCommentId=13071242&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13071242

          Show
          Eric Payne added a comment - I am duplicating HDFS-2171 to this Jira ( HDFS-2202 ) so as to make the review and commit steps easier. HDFS-2171 was originally for 0.20.205 and HDFS-2202 was originally for 0.23. Now, HDFS-2202 is for both Please see Requirements and Design Tradeoffs here: https://issues.apache.org/jira/browse/HDFS-2171?focusedCommentId=13068990&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13068990 Please see junit and manual test information here: https://issues.apache.org/jira/browse/HDFS-2171?focusedCommentId=13071242&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13071242
          Hide
          Eric Payne added a comment -

          Correction: Duplicating this Jira (HDFS-2202) to HDFS-2171.

          Show
          Eric Payne added a comment - Correction: Duplicating this Jira ( HDFS-2202 ) to HDFS-2171 .
          Hide
          Eric Payne added a comment -

          Actually, I had it right the first time. Duplicating HDFS-2171.

          Show
          Eric Payne added a comment - Actually, I had it right the first time. Duplicating HDFS-2171 .
          Eric Payne made changes -
          Fix Version/s 0.20.205.0 [ 12316392 ]
          Affects Version/s 0.23.0 [ 12315571 ]
          Eric Payne made changes -
          Summary 0.23: Changes to balancer bandwidth should not require datanode restart. Changes to balancer bandwidth should not require datanode restart.
          Eric Payne made changes -
          Attachment HDFS-2202.0.23.0.v1.patch [ 12488118 ]
          Attachment HDFS-2202.0.20.205.0.v1.patch [ 12488119 ]
          Hide
          Eric Payne added a comment -

          Hi Nicholas,

          Thank you for reviewing this Jira. Your comments were clear, precise, and easily understood. I appreciate that.

          > Hi Eric, sorry that the refactoring breaks your patch. Could you update it?
          Yes. It has been updated.

          > In TestBalancerBandwidth, you may call MiniDFSCluster.getFileSystem() instead of creating a DFSClient.
          Done.

          > We should update ClientProtocol.versionID and DatanodeProtocol.versionID.
          > I think the BalancerBandwidthCommand.version is not needed. We have to change the DatanodeProtocol.versionID in this case.

          I did this in the 0.23.0 patch. However, one of the requirements for the 0.20.205.0 patch was to not modify the DatanodeProtocol.versionID (please see https://issues.apache.org/jira/browse/HDFS-2171?focusedCommentId=13068990&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13068990). The reason is that the operations team does not want to require all clusters in a colo to be upgraded for 0.20.205, which would have to be done if the DatanodeProtocol.versionID changed. This is because there are some cross-cluster use cases.

          In 0.20.205, I left the BalancerBandwidthCommand.version.
          In the case of 0.23, the DatanodeProtocol.versionID has to change anyway, so it makes sense there.

          > You may use for-each statement for the following (... foreach example code here...)
          Done

          > The initial capacity does not really matter. How about removing it?
          Done

          > Please add getter/setter and do not use public field DatanodeDescriptor.bandwidth.
          Done

          > Please add javadoc (or change comments to javadoc) to all new public classes/methods/fields.
          Done

          Show
          Eric Payne added a comment - Hi Nicholas, Thank you for reviewing this Jira. Your comments were clear, precise, and easily understood. I appreciate that. > Hi Eric, sorry that the refactoring breaks your patch. Could you update it? Yes. It has been updated. > In TestBalancerBandwidth, you may call MiniDFSCluster.getFileSystem() instead of creating a DFSClient. Done. > We should update ClientProtocol.versionID and DatanodeProtocol.versionID. > I think the BalancerBandwidthCommand.version is not needed. We have to change the DatanodeProtocol.versionID in this case. I did this in the 0.23.0 patch. However, one of the requirements for the 0.20.205.0 patch was to not modify the DatanodeProtocol.versionID (please see https://issues.apache.org/jira/browse/HDFS-2171?focusedCommentId=13068990&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13068990 ). The reason is that the operations team does not want to require all clusters in a colo to be upgraded for 0.20.205, which would have to be done if the DatanodeProtocol.versionID changed. This is because there are some cross-cluster use cases. In 0.20.205, I left the BalancerBandwidthCommand.version. In the case of 0.23, the DatanodeProtocol.versionID has to change anyway, so it makes sense there. > You may use for-each statement for the following (... foreach example code here...) Done > The initial capacity does not really matter. How about removing it? Done > Please add getter/setter and do not use public field DatanodeDescriptor.bandwidth. Done > Please add javadoc (or change comments to javadoc) to all new public classes/methods/fields. Done
          Hide
          Tsz Wo Nicholas Sze added a comment -

          All hudson machines are down. Could you run "ant test" and "ant test-patch" manually?

          Show
          Tsz Wo Nicholas Sze added a comment - All hudson machines are down. Could you run "ant test" and "ant test-patch" manually?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          The new patch looks good. Some minor comments:

          • There is no javadoc for the class BalancerBandwidthCommand.
          • Please change the comment above DistributedFileSystem.setBalancerBandwidth(..) to javadoc.
          Show
          Tsz Wo Nicholas Sze added a comment - The new patch looks good. Some minor comments: There is no javadoc for the class BalancerBandwidthCommand. Please change the comment above DistributedFileSystem.setBalancerBandwidth(..) to javadoc.
          Eric Payne made changes -
          Attachment HDFS-2202.0.23.0.v2.patch [ 12488217 ]
          Attachment HDFS-2202.0.20.205.0.v2.patch [ 12488218 ]
          Hide
          Eric Payne added a comment -

          Thanks Nicholas.

          > There is no javadoc for the class BalancerBandwidthCommand.
          Done.

          > Please change the comment above DistributedFileSystem.setBalancerBandwidth(..) to javadoc.
          Done.

          Show
          Eric Payne added a comment - Thanks Nicholas. > There is no javadoc for the class BalancerBandwidthCommand. Done. > Please change the comment above DistributedFileSystem.setBalancerBandwidth(..) to javadoc. Done.
          Hide
          Eric Payne added a comment -

          [exec] +1 overall.
          [exec]
          [exec] +1 @author. The patch does not contain any @author tags.
          [exec]
          [exec] +1 tests included. The patch appears to include 3 new or modified tests.
          [exec]
          [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
          [exec]
          [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
          [exec]
          [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
          [exec]
          [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
          [exec]
          [exec] +1 system test framework. The patch passed system test framework compile.
          [exec]
          [exec]
          [exec]
          [exec]
          [exec] ======================================================================
          [exec] ======================================================================
          [exec] Finished build.
          [exec] ======================================================================
          [exec] ======================================================================
          [exec]
          [exec]

          BUILD SUCCESSFUL

          Show
          Eric Payne added a comment - [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. [exec] [exec] [exec] [exec] [exec] ====================================================================== [exec] ====================================================================== [exec] Finished build. [exec] ====================================================================== [exec] ====================================================================== [exec] [exec] BUILD SUCCESSFUL
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch look good.

          Please also post the "ant test" results.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch look good. Please also post the "ant test" results.
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags [Reviewed]
          Issue Type Bug [ 1 ] New Feature [ 2 ]
          Hide
          Eric Payne added a comment -

          ant test output. attached here because yahoo hudson server is down.

          Show
          Eric Payne added a comment - ant test output. attached here because yahoo hudson server is down.
          Eric Payne made changes -
          Attachment ant.test.0.23.out [ 12488241 ]
          Hide
          Eric Payne added a comment -

          ant test output attached.

          Failures: TestReplaceDataNodeOnFailure: Not related to changes in this Jira. Also,did not fail when run individually
          Error: TestHFlush: Not related to changes in this Jira. Also,did not fail when run individually
          Error: TestDistributedUpgrade: Not related to changes in this Jira. Also,did not fail when run individually

          Show
          Eric Payne added a comment - ant test output attached. Failures: TestReplaceDataNodeOnFailure: Not related to changes in this Jira. Also,did not fail when run individually Error: TestHFlush: Not related to changes in this Jira. Also,did not fail when run individually Error: TestDistributedUpgrade: Not related to changes in this Jira. Also,did not fail when run individually
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Eric!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Eric!
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed] [Incompatible change, Reviewed]
          Release Note New dfsadmin command added: [-setBalancerBandwidth <bandwidth>] where bandwidth is max network bandwidth in bytes per second that the balancer is allowed to use on each datanode during balacing. New dfsadmin command added: [-setBalancerBandwidth <bandwidth>] where bandwidth is max network bandwidth in bytes per second that the balancer is allowed to use on each datanode during balacing.

          This is an incompatible change in 0.23. The versions of ClientProtocol and DatanodeProtocol are changed.
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #738 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/738/)
          HDFS-2202. Add a new DFSAdmin command to set balancer bandwidth of datanodes without restarting. Contributed by Eric Payne

          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152401
          Files :

          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestBalancerBandwidth.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/BalancerBandwidthCommand.java
          • /hadoop/common/trunk/hdfs/CHANGES.txt
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #738 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/738/ ) HDFS-2202 . Add a new DFSAdmin command to set balancer bandwidth of datanodes without restarting. Contributed by Eric Payne szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152401 Files : /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestBalancerBandwidth.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/BalancerBandwidthCommand.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #812 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/812/)
          HDFS-2202. Add a new DFSAdmin command to set balancer bandwidth of datanodes without restarting. Contributed by Eric Payne

          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152401
          Files :

          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestBalancerBandwidth.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/BalancerBandwidthCommand.java
          • /hadoop/common/trunk/hdfs/CHANGES.txt
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #812 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/812/ ) HDFS-2202 . Add a new DFSAdmin command to set balancer bandwidth of datanodes without restarting. Contributed by Eric Payne szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152401 Files : /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestBalancerBandwidth.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/BalancerBandwidthCommand.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
          Eric Payne made changes -
          Attachment Balancer Bandwidth MSC.jpg [ 12489903 ]
          Hide
          Matt Foley added a comment -

          Closed upon release of 0.20.205.0

          Show
          Matt Foley added a comment - Closed upon release of 0.20.205.0
          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Harsh J made changes -
          Link This issue is related to HDFS-2391 [ HDFS-2391 ]
          Harsh J made changes -
          Link This issue is related to HDFS-3331 [ HDFS-3331 ]

            People

            • Assignee:
              Eric Payne
              Reporter:
              Eric Payne
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development