Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3967

Fetch the application report from the AHS if the RM does not know about it

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      If the application history service has been enabled and RM has forgotten anout an application, try and fetch the app report form the AHS.

      On larger clusters, the RM can forget about the applications in about 30 minutes. The proxy url generated during the job submission will try to fetch the app report from the RM and will fail to get anything from there. If the app is not found in the RM, we will need to get the application report from the Application History Server (if it is enabled) to see if we can get any information on that application before throwing an exception.

      1. YARN-3967.1.patch
        11 kB
        Mit Desai
      2. YARN-3967.2.patch
        11 kB
        Mit Desai
      3. YARN-3967.3.patch
        11 kB
        Mit Desai

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2213 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2213/)
        YARN-3967. Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2213 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2213/ ) YARN-3967 . Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #264 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/264/)
        YARN-3967. Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #264 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/264/ ) YARN-3967 . Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #256 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/256/)
        YARN-3967. Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #256 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/256/ ) YARN-3967 . Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2194 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2194/)
        YARN-3967. Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2194 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2194/ ) YARN-3967 . Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #997 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/997/)
        YARN-3967. Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #997 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/997/ ) YARN-3967 . Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #267 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/267/)
        YARN-3967. Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #267 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/267/ ) YARN-3967 . Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        Hide
        mitdesai Mit Desai added a comment -

        Filed YARN-3976

        Show
        mitdesai Mit Desai added a comment - Filed YARN-3976
        Hide
        mitdesai Mit Desai added a comment -

        Agreed. We should make the change in at both places than. I'll file a Jira for that as well.

        Show
        mitdesai Mit Desai added a comment - Agreed. We should make the change in at both places than. I'll file a Jira for that as well.
        Hide
        jlowe Jason Lowe added a comment -

        Thanks for filing the followup.

        This was to keep the implementation consistent with what the YarnClient does. If we want to make the change, I would suggest changing both YarnClient and AppReportFetcher to behave in a similar manner.

        That's unfortunate, since it's a convoluted way to handle the exception. It's much clearer to catch the specific exception rather than the parent exception and rethrow it when it's not the specific one. Even elsewhere in YarnClientImpl it does the right thing and catches ApplicationNotFoundException directly, see YarnClientImpl.submitApplication.

        Show
        jlowe Jason Lowe added a comment - Thanks for filing the followup. This was to keep the implementation consistent with what the YarnClient does. If we want to make the change, I would suggest changing both YarnClient and AppReportFetcher to behave in a similar manner. That's unfortunate, since it's a convoluted way to handle the exception. It's much clearer to catch the specific exception rather than the parent exception and rethrow it when it's not the specific one. Even elsewhere in YarnClientImpl it does the right thing and catches ApplicationNotFoundException directly, see YarnClientImpl.submitApplication.
        Hide
        mitdesai Mit Desai added a comment -
        Show
        mitdesai Mit Desai added a comment - YARN-3975
        Hide
        mitdesai Mit Desai added a comment -

        Created YARN-2975

        Show
        mitdesai Mit Desai added a comment - Created YARN-2975
        Hide
        mitdesai Mit Desai added a comment -

        I'm confused why we are not simply adding a separate explicit clause to catch ApplicationNotFoundException

        Jason Lowe This was to keep the implementation consistent with what the YarnClient does. If we want to make the change, I would suggest changing both YarnClient and AppReportFetcher to behave in a similar manner.

        This code also needs to be updated to handle the case when the appreport doesn't have a tracking URL and we got the report from the history server, as we will direct the user to the RM app page. We already know the RM app page doesn't exist due to the ApplicationNotFoundException earlier:

        Nice catch. This is a rare case that should not happen in practice. I'll file another Jira to fix that issue.

        Show
        mitdesai Mit Desai added a comment - I'm confused why we are not simply adding a separate explicit clause to catch ApplicationNotFoundException Jason Lowe This was to keep the implementation consistent with what the YarnClient does. If we want to make the change, I would suggest changing both YarnClient and AppReportFetcher to behave in a similar manner. This code also needs to be updated to handle the case when the appreport doesn't have a tracking URL and we got the report from the history server, as we will direct the user to the RM app page. We already know the RM app page doesn't exist due to the ApplicationNotFoundException earlier: Nice catch. This is a rare case that should not happen in practice. I'll file another Jira to fix that issue.
        Hide
        jlowe Jason Lowe added a comment -

        I'm confused why we are not simply adding a separate explicit clause to catch ApplicationNotFoundException instead of YarnException. Seems like it should just be:

            try {
              response = applicationsManager.getApplicationReport(request);
            } catch (ApplicationNotFoundException e) {
              if (!isAHSEnabled) {
                // Just throw it as usual if historyService is not enabled.
                throw e;
              }
              response = historyManager.getApplicationReport(request);
            }
        

        This code also needs to be updated to handle the case when the appreport doesn't have a tracking URL and we got the report from the history server, as we will direct the user to the RM app page. We already know the RM app page doesn't exist due to the ApplicationNotFoundException earlier:

              // fallback to ResourceManager's app page if no tracking URI provided
              if(original == null || original.equals("N/A")) {
                resp.sendRedirect(resp.encodeRedirectURL(
                    StringHelper.pjoin(rmAppPageUrlBase, id.toString())));
                return;
              } else {
                if (ProxyUriUtils.getSchemeFromUrl(original).isEmpty()) {
                  trackingUri = ProxyUriUtils.getUriFromAMUrl(
                      WebAppUtils.getHttpSchemePrefix(conf), original);
                } else {
                  trackingUri = new URI(original);
                }
              }
        
        Show
        jlowe Jason Lowe added a comment - I'm confused why we are not simply adding a separate explicit clause to catch ApplicationNotFoundException instead of YarnException. Seems like it should just be: try { response = applicationsManager.getApplicationReport(request); } catch (ApplicationNotFoundException e) { if (!isAHSEnabled) { // Just throw it as usual if historyService is not enabled. throw e; } response = historyManager.getApplicationReport(request); } This code also needs to be updated to handle the case when the appreport doesn't have a tracking URL and we got the report from the history server, as we will direct the user to the RM app page. We already know the RM app page doesn't exist due to the ApplicationNotFoundException earlier: // fallback to ResourceManager's app page if no tracking URI provided if (original == null || original.equals( "N/A" )) { resp.sendRedirect(resp.encodeRedirectURL( StringHelper.pjoin(rmAppPageUrlBase, id.toString()))); return ; } else { if (ProxyUriUtils.getSchemeFromUrl(original).isEmpty()) { trackingUri = ProxyUriUtils.getUriFromAMUrl( WebAppUtils.getHttpSchemePrefix(conf), original); } else { trackingUri = new URI(original); } }
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8215 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8215/)
        YARN-3967. Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8215 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8215/ ) YARN-3967 . Fetch the application report from the AHS if the RM does not (xgong: rev fbd6063269221ec25834684477f434e19f0b66af) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/AppReportFetcher.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestAppReportFetcher.java
        Hide
        xgong Xuan Gong added a comment -

        Committed into trunk/branch-2/branch-2.7. Thanks, Mit Desai.

        Show
        xgong Xuan Gong added a comment - Committed into trunk/branch-2/branch-2.7. Thanks, Mit Desai .
        Hide
        jlowe Jason Lowe added a comment -

        Sorry for the late arrival, can I get a chance to review?

        Show
        jlowe Jason Lowe added a comment - Sorry for the late arrival, can I get a chance to review?
        Hide
        xgong Xuan Gong added a comment -

        Mit Desai Ah, sorry for missing that...

        +1 for the patch, checking this in

        Show
        xgong Xuan Gong added a comment - Mit Desai Ah, sorry for missing that... +1 for the patch, checking this in
        Hide
        hadoopqa Hadoop QA added a comment -



        +1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 15m 25s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 39s There were no new javac warning messages.
        +1 javadoc 9m 38s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 22s There were no new checkstyle issues.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 19s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 0m 43s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 yarn tests 0m 23s Tests passed in hadoop-yarn-server-web-proxy.
            36m 29s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12747052/YARN-3967.3.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 0fcb4a8
        hadoop-yarn-server-web-proxy test log https://builds.apache.org/job/PreCommit-YARN-Build/8651/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8651/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/8651/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 25s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 39s There were no new javac warning messages. +1 javadoc 9m 38s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 22s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 19s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 0m 43s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-server-web-proxy.     36m 29s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747052/YARN-3967.3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0fcb4a8 hadoop-yarn-server-web-proxy test log https://builds.apache.org/job/PreCommit-YARN-Build/8651/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8651/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8651/console This message was automatically generated.
        Hide
        mitdesai Mit Desai added a comment -

        Refined the patch

        Xuan Gong, I actually tried doing that before. But historyManager is a final member and cannot be initialiez outside. So it is not possible to make that change. There are 2 constructors for this class so initialization will have to be in 2 places.

        Show
        mitdesai Mit Desai added a comment - Refined the patch Xuan Gong , I actually tried doing that before. But historyManager is a final member and cannot be initialiez outside. So it is not possible to make that change. There are 2 constructors for this class so initialization will have to be in 2 places.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 56s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 9m 20s There were no new javac warning messages.
        +1 javadoc 12m 3s There were no new javadoc warning messages.
        +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 0m 26s The applied patch generated 5 new checkstyle issues (total was 3, now 8).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 45s mvn install still works.
        +1 eclipse:eclipse 0m 44s The patch built with eclipse:eclipse.
        +1 findbugs 0m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 yarn tests 0m 30s Tests passed in hadoop-yarn-server-web-proxy.
            45m 11s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12746856/YARN-3967.2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 813cf89
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8638/artifact/patchprocess/diffcheckstylehadoop-yarn-server-web-proxy.txt
        hadoop-yarn-server-web-proxy test log https://builds.apache.org/job/PreCommit-YARN-Build/8638/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8638/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/8638/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 56s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 9m 20s There were no new javac warning messages. +1 javadoc 12m 3s There were no new javadoc warning messages. +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 26s The applied patch generated 5 new checkstyle issues (total was 3, now 8). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 45s mvn install still works. +1 eclipse:eclipse 0m 44s The patch built with eclipse:eclipse. +1 findbugs 0m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 30s Tests passed in hadoop-yarn-server-web-proxy.     45m 11s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12746856/YARN-3967.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 813cf89 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8638/artifact/patchprocess/diffcheckstylehadoop-yarn-server-web-proxy.txt hadoop-yarn-server-web-proxy test log https://builds.apache.org/job/PreCommit-YARN-Build/8638/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8638/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8638/console This message was automatically generated.
        Hide
        xgong Xuan Gong added a comment -

        Thanks for working on this. Mit Desai. The patch looks good overall. Just one minor comment:
        Could we create a common function for this

        if (conf.getBoolean(YarnConfiguration.APPLICATION_HISTORY_ENABLED,      YarnConfiguration.DEFAULT_APPLICATION_HISTORY_ENABLED)) {
            isAHSEnabled = true;
        	    }
        
        	    if (isAHSEnabled) {
        	      try {
        	        historyManager = getAHSProxy(conf);
        	      } catch (IOException e) {
        	        throw new YarnRuntimeException(e);
        	      }
        	    }
        	    else {
                      this.historyManager = null;
        

        Since this piece of code is used in multiple place ?

        Show
        xgong Xuan Gong added a comment - Thanks for working on this. Mit Desai . The patch looks good overall. Just one minor comment: Could we create a common function for this if (conf.getBoolean(YarnConfiguration.APPLICATION_HISTORY_ENABLED, YarnConfiguration.DEFAULT_APPLICATION_HISTORY_ENABLED)) { isAHSEnabled = true ; } if (isAHSEnabled) { try { historyManager = getAHSProxy(conf); } catch (IOException e) { throw new YarnRuntimeException(e); } } else { this .historyManager = null ; Since this piece of code is used in multiple place ?
        Hide
        mitdesai Mit Desai added a comment -

        Fixed checkstyle and whitespaces

        Show
        mitdesai Mit Desai added a comment - Fixed checkstyle and whitespaces
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 15m 38s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 46s There were no new javac warning messages.
        +1 javadoc 9m 47s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 0m 22s The applied patch generated 28 new checkstyle issues (total was 2, now 30).
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 21s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 0m 44s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 yarn tests 0m 23s Tests passed in hadoop-yarn-server-web-proxy.
            37m 1s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12746839/YARN-3967.1.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 813cf89
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8637/artifact/patchprocess/diffcheckstylehadoop-yarn-server-web-proxy.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/8637/artifact/patchprocess/whitespace.txt
        hadoop-yarn-server-web-proxy test log https://builds.apache.org/job/PreCommit-YARN-Build/8637/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8637/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/8637/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 38s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 46s There were no new javac warning messages. +1 javadoc 9m 47s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 22s The applied patch generated 28 new checkstyle issues (total was 2, now 30). -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 21s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 0m 44s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-server-web-proxy.     37m 1s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12746839/YARN-3967.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 813cf89 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8637/artifact/patchprocess/diffcheckstylehadoop-yarn-server-web-proxy.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/8637/artifact/patchprocess/whitespace.txt hadoop-yarn-server-web-proxy test log https://builds.apache.org/job/PreCommit-YARN-Build/8637/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8637/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8637/console This message was automatically generated.
        Hide
        jlowe Jason Lowe added a comment -

        Yes, the YARN client has this redirection built into it, but the web proxy isn't using that client.

        Show
        jlowe Jason Lowe added a comment - Yes, the YARN client has this redirection built into it, but the web proxy isn't using that client.
        Hide
        xgong Xuan Gong added a comment -

        At least for the YARN CLI, i think that right now we have this redirection.

        Show
        xgong Xuan Gong added a comment - At least for the YARN CLI, i think that right now we have this redirection.
        Hide
        mitdesai Mit Desai added a comment -

        Attaching the patch

        Show
        mitdesai Mit Desai added a comment - Attaching the patch

          People

          • Assignee:
            mitdesai Mit Desai
            Reporter:
            mitdesai Mit Desai
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development