Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3358

Specify explicitly that the NN UI status total is talking of persistent objects on heap.

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 3.0.0
    • Component/s: namenode
    • Labels:
      None

      Description

      The NN shows, on its web UI, something like:

      223 files and directories, 138 blocks = 361 total.

      Followed by heap stats.

      We should clarify this line is talking of objects and is related to the heap summaries. Perhaps just being explicit about java-terms would be nicer:

      223 files and directories, 138 blocks = 361 total objects.

      1. HDFS-3358.patch
        3 kB
        Harsh J
      2. HDFS-3358.patch
        3 kB
        Harsh J
      3. HDFS-3358.patch
        3 kB
        Harsh J

        Activity

        Hide
        Aaron T. Myers added a comment -

        Harsh, mind committing this to branch-2 as well? Seems like a pretty low-risk improvement to me.

        Show
        Aaron T. Myers added a comment - Harsh, mind committing this to branch-2 as well? Seems like a pretty low-risk improvement to me.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1276 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1276/)
        HDFS-3358. Specify explicitly that the NN UI status total is talking of persistent objects on heap. Contributed by Harsh J. (harsh) (Revision 1416245)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1416245
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestMetaSave.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1276 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1276/ ) HDFS-3358 . Specify explicitly that the NN UI status total is talking of persistent objects on heap. Contributed by Harsh J. (harsh) (Revision 1416245) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1416245 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestMetaSave.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1245 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1245/)
        HDFS-3358. Specify explicitly that the NN UI status total is talking of persistent objects on heap. Contributed by Harsh J. (harsh) (Revision 1416245)

        Result = FAILURE
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1416245
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestMetaSave.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1245 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1245/ ) HDFS-3358 . Specify explicitly that the NN UI status total is talking of persistent objects on heap. Contributed by Harsh J. (harsh) (Revision 1416245) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1416245 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestMetaSave.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #55 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/55/)
        HDFS-3358. Specify explicitly that the NN UI status total is talking of persistent objects on heap. Contributed by Harsh J. (harsh) (Revision 1416245)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1416245
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestMetaSave.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #55 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/55/ ) HDFS-3358 . Specify explicitly that the NN UI status total is talking of persistent objects on heap. Contributed by Harsh J. (harsh) (Revision 1416245) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1416245 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestMetaSave.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3082 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3082/)
        HDFS-3358. Specify explicitly that the NN UI status total is talking of persistent objects on heap. Contributed by Harsh J. (harsh) (Revision 1416245)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1416245
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestMetaSave.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3082 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3082/ ) HDFS-3358 . Specify explicitly that the NN UI status total is talking of persistent objects on heap. Contributed by Harsh J. (harsh) (Revision 1416245) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1416245 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestMetaSave.java
        Hide
        Harsh J added a comment -

        Thanks Suresh! Committed to trunk.

        Show
        Harsh J added a comment - Thanks Suresh! Committed to trunk.
        Hide
        Suresh Srinivas added a comment -

        +1 for the change.

        Show
        Suresh Srinivas added a comment - +1 for the change.
        Hide
        Harsh J added a comment -

        Any further comments on this trivial messaging change?

        Show
        Harsh J added a comment - Any further comments on this trivial messaging change?
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12548175/HDFS-3358.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3287//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3287//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548175/HDFS-3358.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3287//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3287//console This message is automatically generated.
        Hide
        Harsh J added a comment -

        Rebased patch that does a "s/total/total filesystem objects" for the shown value (i.e. no change from previous patch, to stay in tune with the configurable max (which is also displayed as a percentaged value if enabled already).

        Show
        Harsh J added a comment - Rebased patch that does a "s/total/total filesystem objects" for the shown value (i.e. no change from previous patch, to stay in tune with the configurable max (which is also displayed as a percentaged value if enabled already).
        Hide
        Harsh J added a comment -

        I mean that are there other similar changes required?

        I do also feel that the heap stats display looks to ambiguous and may be improved upon, but otherwise no, no other similar changes are required here.

        I'll revise the patch to match the config and if it looks good to most here, I'll add it in. Thanks guys!

        Show
        Harsh J added a comment - I mean that are there other similar changes required? I do also feel that the heap stats display looks to ambiguous and may be improved upon, but otherwise no, no other similar changes are required here. I'll revise the patch to match the config and if it looks good to most here, I'll add it in. Thanks guys!
        Hide
        Tsz Wo Nicholas Sze added a comment -

        As mentioned previously, it does not seem helpful to add the "objects". In such case, I rather keep the UI unchanged.

        ... I'd prefer if we rather just printed "X inodes"

        Would INode be too technical to users?

        bq Are there similar changes on the web UI?

        Not sure I understood this one.

        I mean that are there other similar changes required? I don't want that we change this line today and then another line tomorrow.

        Show
        Tsz Wo Nicholas Sze added a comment - As mentioned previously, it does not seem helpful to add the "objects". In such case, I rather keep the UI unchanged. ... I'd prefer if we rather just printed "X inodes" Would INode be too technical to users? bq Are there similar changes on the web UI? Not sure I understood this one. I mean that are there other similar changes required? I don't want that we change this line today and then another line tomorrow.
        Hide
        Harsh J added a comment -

        And when that config can use objects in its wording, why can't we just use the same on the UI to be consistent?

        Show
        Harsh J added a comment - And when that config can use objects in its wording, why can't we just use the same on the UI to be consistent?
        Hide
        Harsh J added a comment -

        Actually, this count is also related to DFSConfigKeys.DFS_NAMENODE_MAX_OBJECTS_KEY, which acts as a per-NN total object limit (seems heap related for sure, if we limit on objects and not individual elements).

        Show
        Harsh J added a comment - Actually, this count is also related to DFSConfigKeys.DFS_NAMENODE_MAX_OBJECTS_KEY, which acts as a per-NN total object limit (seems heap related for sure, if we limit on objects and not individual elements).
        Hide
        Harsh J added a comment -

        Should we include symlinks?

        Instead of making it "X files, directories and symlinks" and given that symlinks are a purely optional usage, I'd prefer if we rather just printed "X inodes"

        Should we separate the counts for files, directories and symlinks?

        This would be lovely to have instead of inode counts. Is there such metrics already available?

        Should we handle singular/plural? "1 blocks" does not look good.

        Yes we should handle this. I'll update this logic in the patch I'll put up next.

        bq Are there similar changes on the web UI?

        Not sure I understood this one.

        We should also refactor the code, so that the same string is not constructed in two different places.

        Agreed and will do.

        Do lemme know thoughts on (1) and possibility of (2)!

        Show
        Harsh J added a comment - Should we include symlinks? Instead of making it "X files, directories and symlinks" and given that symlinks are a purely optional usage, I'd prefer if we rather just printed "X inodes" Should we separate the counts for files, directories and symlinks? This would be lovely to have instead of inode counts. Is there such metrics already available? Should we handle singular/plural? "1 blocks" does not look good. Yes we should handle this. I'll update this logic in the patch I'll put up next. bq Are there similar changes on the web UI? Not sure I understood this one. We should also refactor the code, so that the same string is not constructed in two different places. Agreed and will do. Do lemme know thoughts on (1) and possibility of (2)!
        Hide
        Tsz Wo Nicholas Sze added a comment -

        If we really want to change the line, we should think about it more carefully:

        • Should we include symlinks?
        • Should we separate the counts for files, directories and symlinks?
        • Should we handle singular/plural? "1 blocks" does not look good.
        • Are there similar changes on the web UI?

        We should also refactor the code, so that the same string is not constructed in two different places.

        Show
        Tsz Wo Nicholas Sze added a comment - If we really want to change the line, we should think about it more carefully: Should we include symlinks? Should we separate the counts for files, directories and symlinks? Should we handle singular/plural? "1 blocks" does not look good. Are there similar changes on the web UI? We should also refactor the code, so that the same string is not constructed in two different places.
        Hide
        Aaron T. Myers added a comment -

        +1, the patch looks good to me. I agree that this makes the UI a little clearer.

        Nicholas/Suresh - does it seem OK to you?

        Show
        Aaron T. Myers added a comment - +1, the patch looks good to me. I agree that this makes the UI a little clearer. Nicholas/Suresh - does it seem OK to you?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12525565/HDFS-3358.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2374//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2374//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525565/HDFS-3358.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2374//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2374//console This message is automatically generated.
        Hide
        Harsh J added a comment -

        TestMetaSave changes pass locally and patch compiles properly. Submitting patch for QA. I didn't see any other patch/features looking for these strings, but might as well run all tests.

        Show
        Harsh J added a comment - TestMetaSave changes pass locally and patch compiles properly. Submitting patch for QA. I didn't see any other patch/features looking for these strings, but might as well run all tests.
        Hide
        Harsh J added a comment -

        I'll choose the second, explicitly-constructive way. Please let me know if this is also vague (would also appreciate your ideas on being explicit properly).

        Show
        Harsh J added a comment - I'll choose the second, explicitly-constructive way. Please let me know if this is also vague (would also appreciate your ideas on being explicit properly).
        Hide
        Harsh J added a comment -

        Nicholas,

        Good question. Will make sure next time I also capture motives. My bad (again!)

        I opened this ticket after what seemed like the 4th time I needed to explain this to users who asked me this. They all get the files and directories, they get the blocks. They confuse themselves with the total. If they are asking so frequently, there's surely something they don't understand about why we've totaled that.

        I have two new proposals in making it easier to read such a line:

        • We remove totals in the lines, as it adds no real value.
        • We prepend or append the line with "Filesystem objects", to indicate the whole line is about some FS objects (With fs being the context they need to read it in).
          • Additionally, but optionally, we may prepend the wide heap stats area with "Memory status:" to signify its purpose, as well as keep it separate in relationship with the other parts.

        The purpose of this JIRA was only to improve some text, not to simply make an 'arbitrary change'. That I wasn't clear on why I bought this up all of a sudden was my fault, and that shall not happen again.

        If we feel strongly no change is required or would make any benefit at all whatsoever, we may also close this out as Invalid

        Show
        Harsh J added a comment - Nicholas, Good question. Will make sure next time I also capture motives. My bad (again!) I opened this ticket after what seemed like the 4th time I needed to explain this to users who asked me this. They all get the files and directories , they get the blocks . They confuse themselves with the total . If they are asking so frequently, there's surely something they don't understand about why we've totaled that. I have two new proposals in making it easier to read such a line: We remove totals in the lines, as it adds no real value. We prepend or append the line with "Filesystem objects", to indicate the whole line is about some FS objects (With fs being the context they need to read it in). Additionally, but optionally, we may prepend the wide heap stats area with "Memory status:" to signify its purpose, as well as keep it separate in relationship with the other parts. The purpose of this JIRA was only to improve some text, not to simply make an 'arbitrary change'. That I wasn't clear on why I bought this up all of a sudden was my fault, and that shall not happen again. If we feel strongly no change is required or would make any benefit at all whatsoever, we may also close this out as Invalid
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Harsh,

        I believe that this was not the first time you saw the UI. Why you think that "223 files and directories, 138 blocks = 361 total" is not clear at this moment?

        > A "total" of what?

        It is understood that it is a total of files, directories and blocks.

        Show
        Tsz Wo Nicholas Sze added a comment - Harsh, I believe that this was not the first time you saw the UI. Why you think that "223 files and directories, 138 blocks = 361 total" is not clear at this moment? > A "total" of what? It is understood that it is a total of files, directories and blocks.
        Hide
        Harsh J added a comment -

        I get that already Suresh. I should ask more clearly, sorry:

        223 files and directories, 138 blocks = 361 total.

        I need to simply know WHY do we add blocks and inodes here?

        Showing counts is one thing, and I think its OK to show counts. Why add counts and call it something vague like "total". A "total" of what?

        Should it matter to an admin what the "total" is? If so, how should he look at it?

        Show
        Harsh J added a comment - I get that already Suresh. I should ask more clearly, sorry: 223 files and directories, 138 blocks = 361 total. I need to simply know WHY do we add blocks and inodes here? Showing counts is one thing, and I think its OK to show counts. Why add counts and call it something vague like "total". A "total" of what? Should it matter to an admin what the "total" is? If so, how should he look at it?
        Hide
        Suresh Srinivas added a comment -

        223 files and directories, 138 blocks = 361 total. Followed by heap stats.

        I have hard time understanding what the confusion is.

        One stat is showing file system related counts. The heap size is java heap size. Yes, the files, dirs and blocks count towards the heap size, so does numerous other objects. These two lines show separate stats about HDFS.

        Show
        Suresh Srinivas added a comment - 223 files and directories, 138 blocks = 361 total. Followed by heap stats. I have hard time understanding what the confusion is. One stat is showing file system related counts. The heap size is java heap size. Yes, the files, dirs and blocks count towards the heap size, so does numerous other objects. These two lines show separate stats about HDFS.
        Hide
        Harsh J added a comment -

        Suppose someone does not understand what does it mean by "223 files and directories, 138 blocks = 361 total." It would be hard to believe that the statement suddenly becomes clear to him/her if the magic word "objects" is added.

        Well it wouldn't be clearer either way, seems like. I welcome your suggestion on how to de-confuse in this case. I merely thought if we add in "object", admins can stop focusing on trying to understand what it means (without gathering understanding first).

        How about "filesystem objects" instead of "objects" then?

        You also said that "Perhaps just being explicit about java-terms would be nicer ..." So, are you using a Java term but not talking about Java?

        Heap's a pretty specific java (well, programming?) term. In accordance with that, adding "objects" would help users ignore that. But yeah, my changes could have caused them to instead go check jmap for object instances too, I guess. My bad in communicating here

        Show
        Harsh J added a comment - Suppose someone does not understand what does it mean by "223 files and directories, 138 blocks = 361 total." It would be hard to believe that the statement suddenly becomes clear to him/her if the magic word "objects" is added. Well it wouldn't be clearer either way, seems like. I welcome your suggestion on how to de-confuse in this case. I merely thought if we add in "object", admins can stop focusing on trying to understand what it means (without gathering understanding first). How about "filesystem objects" instead of "objects" then? You also said that "Perhaps just being explicit about java-terms would be nicer ..." So, are you using a Java term but not talking about Java? Heap's a pretty specific java (well, programming?) term. In accordance with that, adding "objects" would help users ignore that. But yeah, my changes could have caused them to instead go check jmap for object instances too, I guess. My bad in communicating here
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Hi Harsh,

        Suppose someone does not understand what does it mean by "223 files and directories, 138 blocks = 361 total." It would be hard to believe that the statement suddenly becomes clear to him/her if the magic word "objects" is added.

        > ... I wasn't talking java objects, ...

        You also said that "Perhaps just being explicit about java-terms would be nicer ..." So, are you using a Java term but not talking about Java?

        Show
        Tsz Wo Nicholas Sze added a comment - Hi Harsh, Suppose someone does not understand what does it mean by "223 files and directories, 138 blocks = 361 total." It would be hard to believe that the statement suddenly becomes clear to him/her if the magic word "objects" is added. > ... I wasn't talking java objects, ... You also said that "Perhaps just being explicit about java-terms would be nicer ..." So, are you using a Java term but not talking about Java?
        Hide
        Harsh J added a comment -

        Thanks atm. Cancelling patch until I can address Suresh's and Nicholas' comments on improving this (and perhaps other aspects of the UI frontpage, which looks like TMI right now to me).

        Show
        Harsh J added a comment - Thanks atm. Cancelling patch until I can address Suresh's and Nicholas' comments on improving this (and perhaps other aspects of the UI frontpage, which looks like TMI right now to me).
        Hide
        Harsh J added a comment -

        I think the current UI is already very clear. We should not make arbitrary changes.

        I understand the last point. Consider the patch cancelled until addressed.

        But the UI is not clear. Its of no help telling X files and directories in the first place, and then adding blocks to the equation just like that. And then having heap sizes reported right after it to indicate its related to that.

        If anything, files # and dirs # should be noted separately, which is also separately computable (but not kept computed afaik). These things may make sense to a HDFS-dev, but has never made enough sense to users, speaking frankly.

        What would be a better statement users (and especially, non-java users, i.e. admins) can understand?

        And if its not for NN's persistent objects (I wasn't talking java objects, I know each of these is a structure/collection, and on the whole they're still an object) we print that, what do we print it for AND add it to some total?

        Show
        Harsh J added a comment - I think the current UI is already very clear. We should not make arbitrary changes. I understand the last point. Consider the patch cancelled until addressed. But the UI is not clear. Its of no help telling X files and directories in the first place, and then adding blocks to the equation just like that. And then having heap sizes reported right after it to indicate its related to that. If anything, files # and dirs # should be noted separately, which is also separately computable (but not kept computed afaik). These things may make sense to a HDFS-dev, but has never made enough sense to users, speaking frankly. What would be a better statement users (and especially, non-java users, i.e. admins) can understand? And if its not for NN's persistent objects (I wasn't talking java objects, I know each of these is a structure/collection, and on the whole they're still an object) we print that, what do we print it for AND add it to some total?
        Hide
        Tsz Wo Nicholas Sze added a comment -

        We should clarify this line is talking of objects and is related to the heap summaries. Perhaps just being explicit about java-terms would be nicer:

        223 files and directories, 138 blocks = 361 total objects.

        It does not seem correct. A single file/directory/block consists of multiple Java objects.

        I think the current UI is already very clear. We should not make arbitrary changes.

        Show
        Tsz Wo Nicholas Sze added a comment - We should clarify this line is talking of objects and is related to the heap summaries. Perhaps just being explicit about java-terms would be nicer: 223 files and directories, 138 blocks = 361 total objects. It does not seem correct. A single file/directory/block consists of multiple Java objects. I think the current UI is already very clear. We should not make arbitrary changes.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12525410/HDFS-3358.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2366//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/2366//artifact/trunk/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2366//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525410/HDFS-3358.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2366//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/2366//artifact/trunk/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2366//console This message is automatically generated.
        Hide
        Suresh Srinivas added a comment -

        I am not sure this is adding any value. Why does adding objects make is more clearer?

        Show
        Suresh Srinivas added a comment - I am not sure this is adding any value. Why does adding objects make is more clearer?
        Hide
        Aaron T. Myers added a comment -

        Marking patch available for Harsh.

        Show
        Aaron T. Myers added a comment - Marking patch available for Harsh.
        Hide
        Harsh J added a comment -

        Patch adds in an 'object' keyword. Other improvements/suggestions are welcome!

        Show
        Harsh J added a comment - Patch adds in an 'object' keyword. Other improvements/suggestions are welcome!

          People

          • Assignee:
            Harsh J
            Reporter:
            Harsh J
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development