Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3390

DFSAdmin should print full stack traces of errors when DEBUG logging is enabled

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      If an error is encountered when running an `hdfs dfsadmin ...' command, only the exception's message is output. It would be handy for debugging if the full stack trace of the exception were output when DEBUG logging is enabled.

      1. HDFS-3390.patch
        2 kB
        Aaron T. Myers
      2. HDFS-3390.patch
        6 kB
        Aaron T. Myers

        Activity

        Hide
        Aaron T. Myers added a comment -

        Here's a patch which addresses the issue. It catches all exceptions from DFSAdmin commands, logs the full stack trace of the exception if LOG.isDebugEnabled, and then re-throws the exception in all cases.

        I tested this manually by setting HADOOP_ROOT_LOGGER=DEBUG,console and running a few invalid commands, e.g. with illegal arguments or non-HDFS URIs.

        Show
        Aaron T. Myers added a comment - Here's a patch which addresses the issue. It catches all exceptions from DFSAdmin commands, logs the full stack trace of the exception if LOG.isDebugEnabled, and then re-throws the exception in all cases. I tested this manually by setting HADOOP_ROOT_LOGGER=DEBUG,console and running a few invalid commands, e.g. with illegal arguments or non-HDFS URIs.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526045/HDFS-3390.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
        org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2392//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2392//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526045/HDFS-3390.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2392//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2392//console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        The double try-catch makes the code hard to read. How about either

        • print the debug message in individual catch block, or
        • set assign the exception to a local variable and the print the debug message outside before the return?
        Show
        Tsz Wo Nicholas Sze added a comment - The double try-catch makes the code hard to read. How about either print the debug message in individual catch block, or set assign the exception to a local variable and the print the debug message outside before the return?
        Hide
        Aaron T. Myers added a comment -

        Thanks a lot for the review, Nicholas.

        The reason I did it the way I did in the original patch is that the DFSAdmin code currently has multiple separate catch blocks, and thus some code would need to be repeated in each one. In the attached patch, I've assigned the exception to a variable and only printed it out in one place at the end. I'm not sure I agree that this is much clearer, but I don't feel strongly about it. If you think this is better, then that's fine by me.

        I strongly suspect that the test failures are unrelated to this patch.

        Show
        Aaron T. Myers added a comment - Thanks a lot for the review, Nicholas. The reason I did it the way I did in the original patch is that the DFSAdmin code currently has multiple separate catch blocks, and thus some code would need to be repeated in each one. In the attached patch, I've assigned the exception to a variable and only printed it out in one place at the end. I'm not sure I agree that this is much clearer, but I don't feel strongly about it. If you think this is better, then that's fine by me. I strongly suspect that the test failures are unrelated to this patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526090/HDFS-3390.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2394//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2394//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526090/HDFS-3390.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2394//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2394//console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good.

        Show
        Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2213 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2213/)
        HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2213 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2213/ ) HDFS-3390 . DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Aaron T. Myers added a comment -

        Thanks a lot for the reviews, Nicholas. I've just committed this to trunk and branch-2.

        Show
        Aaron T. Myers added a comment - Thanks a lot for the reviews, Nicholas. I've just committed this to trunk and branch-2.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2288 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2288/)
        HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2288 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2288/ ) HDFS-3390 . DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2230/)
        HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324)

        Result = ABORTED
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2230/ ) HDFS-3390 . DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324) Result = ABORTED atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1075 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1075/)
        HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1075 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1075/ ) HDFS-3390 . DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1040 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1040/)
        HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324)

        Result = FAILURE
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1040 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1040/ ) HDFS-3390 . DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java

          People

          • Assignee:
            Aaron T. Myers
            Reporter:
            Aaron T. Myers
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development