Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3390

DFSAdmin should print full stack traces of errors when DEBUG logging is enabled

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      If an error is encountered when running an `hdfs dfsadmin ...' command, only the exception's message is output. It would be handy for debugging if the full stack trace of the exception were output when DEBUG logging is enabled.

      1. HDFS-3390.patch
        2 kB
        Aaron T. Myers
      2. HDFS-3390.patch
        6 kB
        Aaron T. Myers

        Activity

        Aaron T. Myers created issue -
        Hide
        Aaron T. Myers added a comment -

        Here's a patch which addresses the issue. It catches all exceptions from DFSAdmin commands, logs the full stack trace of the exception if LOG.isDebugEnabled, and then re-throws the exception in all cases.

        I tested this manually by setting HADOOP_ROOT_LOGGER=DEBUG,console and running a few invalid commands, e.g. with illegal arguments or non-HDFS URIs.

        Show
        Aaron T. Myers added a comment - Here's a patch which addresses the issue. It catches all exceptions from DFSAdmin commands, logs the full stack trace of the exception if LOG.isDebugEnabled, and then re-throws the exception in all cases. I tested this manually by setting HADOOP_ROOT_LOGGER=DEBUG,console and running a few invalid commands, e.g. with illegal arguments or non-HDFS URIs.
        Aaron T. Myers made changes -
        Field Original Value New Value
        Attachment HDFS-3390.patch [ 12526045 ]
        Aaron T. Myers made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526045/HDFS-3390.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
        org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2392//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2392//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526045/HDFS-3390.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2392//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2392//console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        The double try-catch makes the code hard to read. How about either

        • print the debug message in individual catch block, or
        • set assign the exception to a local variable and the print the debug message outside before the return?
        Show
        Tsz Wo Nicholas Sze added a comment - The double try-catch makes the code hard to read. How about either print the debug message in individual catch block, or set assign the exception to a local variable and the print the debug message outside before the return?
        Hide
        Aaron T. Myers added a comment -

        Thanks a lot for the review, Nicholas.

        The reason I did it the way I did in the original patch is that the DFSAdmin code currently has multiple separate catch blocks, and thus some code would need to be repeated in each one. In the attached patch, I've assigned the exception to a variable and only printed it out in one place at the end. I'm not sure I agree that this is much clearer, but I don't feel strongly about it. If you think this is better, then that's fine by me.

        I strongly suspect that the test failures are unrelated to this patch.

        Show
        Aaron T. Myers added a comment - Thanks a lot for the review, Nicholas. The reason I did it the way I did in the original patch is that the DFSAdmin code currently has multiple separate catch blocks, and thus some code would need to be repeated in each one. In the attached patch, I've assigned the exception to a variable and only printed it out in one place at the end. I'm not sure I agree that this is much clearer, but I don't feel strongly about it. If you think this is better, then that's fine by me. I strongly suspect that the test failures are unrelated to this patch.
        Aaron T. Myers made changes -
        Attachment HDFS-3390.patch [ 12526090 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526090/HDFS-3390.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2394//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2394//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526090/HDFS-3390.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2394//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2394//console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good.

        Show
        Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
        Tsz Wo Nicholas Sze made changes -
        Hadoop Flags Reviewed [ 10343 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2213 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2213/)
        HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2213 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2213/ ) HDFS-3390 . DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Aaron T. Myers added a comment -

        Thanks a lot for the reviews, Nicholas. I've just committed this to trunk and branch-2.

        Show
        Aaron T. Myers added a comment - Thanks a lot for the reviews, Nicholas. I've just committed this to trunk and branch-2.
        Aaron T. Myers made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 2.0.0 [ 12320353 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2288 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2288/)
        HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2288 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2288/ ) HDFS-3390 . DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2230/)
        HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324)

        Result = ABORTED
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2230/ ) HDFS-3390 . DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324) Result = ABORTED atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1075 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1075/)
        HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1075 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1075/ ) HDFS-3390 . DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1040 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1040/)
        HDFS-3390. DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324)

        Result = FAILURE
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1040 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1040/ ) HDFS-3390 . DFSAdmin should print full stack traces of errors when DEBUG logging is enabled. Contributed by Aaron T. Myers. (Revision 1336324) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336324 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Todd Lipcon made changes -
        Fix Version/s 2.0.1 [ 12321440 ]
        Fix Version/s 2.0.0 [ 12320353 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Arun C Murthy made changes -
        Fix Version/s 2.0.2-alpha [ 12322472 ]
        Fix Version/s 2.1.0-alpha [ 12321440 ]

          People

          • Assignee:
            Aaron T. Myers
            Reporter:
            Aaron T. Myers
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development