Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 0.24.0
    • Fix Version/s: 0.23.0, 0.24.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

            final List<DatanodeDescriptor> live = new ArrayList<DatanodeDescriptor>();
            final List<DatanodeDescriptor> dead = new ArrayList<DatanodeDescriptor>();
            blockManager.getDatanodeManager().fetchDatanodes(live, dead, false);
            out.println("Live Datanodes: "+live.size());
            out.println("Dead Datanodes: "+dead.size());
            blockManager.metaSave(out);
      

      Logically all the dataNode related logic can be moved to BlockManager.

      So, here metaSave API is getting the DataNodes details from blockManager to FsNameSystem and after printing the details again it is giving the control to blockManager.

      It will be good to move the DNs size printing code also to BlockManager.

      1. HDFS-2363.patch
        2 kB
        Uma Maheswara Rao G

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #23 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/23/)
        svn merge -c 1176733 from trunk for HDFS-2363.

        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176737
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #23 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/23/ ) svn merge -c 1176733 from trunk for HDFS-2363 . szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176737 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #844 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/844/)
        HDFS-2363. Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G

        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176733
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #844 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/844/ ) HDFS-2363 . Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176733 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #30 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/30/)
        svn merge -c 1176733 from trunk for HDFS-2363.

        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176737
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #30 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/30/ ) svn merge -c 1176733 from trunk for HDFS-2363 . szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176737 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #814 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/814/)
        HDFS-2363. Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G

        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176733
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #814 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/814/ ) HDFS-2363 . Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176733 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1053 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1053/)
        HDFS-2363. Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G

        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176733
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1053 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1053/ ) HDFS-2363 . Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176733 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #995 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/995/)
        HDFS-2363. Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G

        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176733
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #995 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/995/ ) HDFS-2363 . Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176733 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have committed this. Thanks, Uma!

        Show
        Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Uma!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #974 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/974/)
        HDFS-2363. Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G

        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176733
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #974 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/974/ ) HDFS-2363 . Move datanodes size printing from FSNamesystem.metasave(..) to BlockManager. Contributed by Uma Maheswara Rao G szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176733 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 good catch.

        Show
        Tsz Wo Nicholas Sze added a comment - +1 good catch.
        Hide
        Uma Maheswara Rao G added a comment -

        Test failures are not related to this patch!
        Since this is re-factoring, not included new tests. Existing Tests should ensure functionality.

        Show
        Uma Maheswara Rao G added a comment - Test failures are not related to this patch! Since this is re-factoring, not included new tests. Existing Tests should ensure functionality.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12496480/HDFS-2363.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.TestDfsOverAvroRpc
        org.apache.hadoop.hdfs.server.blockmanagement.TestHost2NodesMap
        org.apache.hadoop.hdfs.server.datanode.TestReplicasMap

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1289//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1289//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496480/HDFS-2363.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestDfsOverAvroRpc org.apache.hadoop.hdfs.server.blockmanagement.TestHost2NodesMap org.apache.hadoop.hdfs.server.datanode.TestReplicasMap +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1289//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1289//console This message is automatically generated.
        Hide
        Uma Maheswara Rao G added a comment -

        Updated patch for review!

        Show
        Uma Maheswara Rao G added a comment - Updated patch for review!

          People

          • Assignee:
            Uma Maheswara Rao G
            Reporter:
            Uma Maheswara Rao G
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development