Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6442

Stack trace is missing when error occurs in client protocol provider's constructor

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: client
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      when provider creation fail dump the stack trace rather than just print out the message

      1. MAPREDUCE-6442.2.patch
        0.9 kB
        Chang Li
      2. MAPREDUCE-6442.patch
        0.9 kB
        Chang Li

        Activity

        Hide
        lichangleo Chang Li added a comment -

        Jason Lowe please help review. Thanks!

        Show
        lichangleo Chang Li added a comment - Jason Lowe please help review. Thanks!
        Hide
        jlowe Jason Lowe added a comment -

        Did you test this patch? This patch will not provide the stack trace, it will basically do the same thing as before, since e.toString() and e.getMessage() are almost the same thing. This should call the String,Throwable argument form of logging hat dumps exception traces instead of the single argument String form.

        Show
        jlowe Jason Lowe added a comment - Did you test this patch? This patch will not provide the stack trace, it will basically do the same thing as before, since e.toString() and e.getMessage() are almost the same thing. This should call the String,Throwable argument form of logging hat dumps exception traces instead of the single argument String form.
        Hide
        lichangleo Chang Li added a comment -

        Jason Lowe thanks for review. Updated my patch.

        Show
        lichangleo Chang Li added a comment - Jason Lowe thanks for review. Updated my patch.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 16s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 51s There were no new javac warning messages.
        +1 javadoc 9m 54s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 46s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 23s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 mapreduce tests 1m 47s Tests passed in hadoop-mapreduce-client-core.
            40m 22s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12748483/MAPREDUCE-6442.2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 469cfcd
        hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5927/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5927/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5927/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 16s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 51s There were no new javac warning messages. +1 javadoc 9m 54s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 46s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 23s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 1m 47s Tests passed in hadoop-mapreduce-client-core.     40m 22s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748483/MAPREDUCE-6442.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 469cfcd hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5927/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5927/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5927/console This message was automatically generated.
        Hide
        jlowe Jason Lowe added a comment -

        Latest patch looks good. Was this one tested?

        Show
        jlowe Jason Lowe added a comment - Latest patch looks good. Was this one tested?
        Hide
        lichangleo Chang Li added a comment -

        Jason Lowe thanks for review! I test by pass a null conf to initialize cluster and the log showed the stack trace

        Failed to use org.apache.hadoop.mapred.YarnClientProtocolProvider due to error:
        java.lang.NullPointerException
                at org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:33)
                at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
                at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
                at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
        
        Show
        lichangleo Chang Li added a comment - Jason Lowe thanks for review! I test by pass a null conf to initialize cluster and the log showed the stack trace Failed to use org.apache.hadoop.mapred.YarnClientProtocolProvider due to error: java.lang.NullPointerException at org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:33) at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95) at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82) at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        Chang Li Thanks for your great work. How about adding a test for for checking the error message? Its test should be added as hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestCluster.java.

        Show
        ozawa Tsuyoshi Ozawa added a comment - Chang Li Thanks for your great work. How about adding a test for for checking the error message? Its test should be added as hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestCluster.java.
        Hide
        lichangleo Chang Li added a comment -

        Hi Tsuyoshi Ozawa, thanks for review. But usually log based change doesn't need a unit test. Also for my change

        catch (Exception e) {
                  LOG.info("Failed to use " + provider.getClass().getName()
                      + " due to error: ", e);
                }
        

        that exception is caught, and error message merely gets logged. The exception is not thrown. Do you have any recommendation how to test that in a unit test?
        Moreover, I have manually test this change. I verified that the stack trace is printed and I post the stack trace in the above comment.

        Show
        lichangleo Chang Li added a comment - Hi Tsuyoshi Ozawa , thanks for review. But usually log based change doesn't need a unit test. Also for my change catch (Exception e) { LOG.info( "Failed to use " + provider.getClass().getName() + " due to error: " , e); } that exception is caught, and error message merely gets logged. The exception is not thrown. Do you have any recommendation how to test that in a unit test? Moreover, I have manually test this change. I verified that the stack trace is printed and I post the stack trace in the above comment.
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        Chang Li Oh, I see. I confirmed that it works well too. +1, checking this in.

        Show
        ozawa Tsuyoshi Ozawa added a comment - Chang Li Oh, I see. I confirmed that it works well too. +1, checking this in.
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        Committed this to trunk, branch-2, branch-2.7. Thanks Chang Li for your contribution and reporting and thanks Jason Lowe for your review.

        Show
        ozawa Tsuyoshi Ozawa added a comment - Committed this to trunk, branch-2, branch-2.7. Thanks Chang Li for your contribution and reporting and thanks Jason Lowe for your review.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #8408 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8408/)
        MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #8408 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8408/ ) MAPREDUCE-6442 . Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #356 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/356/)
        MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #356 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/356/ ) MAPREDUCE-6442 . Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2299 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2299/)
        MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2299 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2299/ ) MAPREDUCE-6442 . Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #1088 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1088/)
        MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1088 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1088/ ) MAPREDUCE-6442 . Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #350 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/350/)
        MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #350 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/350/ ) MAPREDUCE-6442 . Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #338 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/338/)
        MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #338 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/338/ ) MAPREDUCE-6442 . Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #2277 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2277/)
        MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2277 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2277/ ) MAPREDUCE-6442 . Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. (ozawa: rev 9b685773ec1031e90cc2b8aedb42670721757e22) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java

          People

          • Assignee:
            lichangleo Chang Li
            Reporter:
            lichangleo Chang Li
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development