Details

    • Target Version/s:

      Description

      Allow for pluggable UIs as described by TEZ-8. Yarn can provide the infrastructure to host java script and possible java UIs.

      1. YARN-2513.v3.patch
        7 kB
        Jonathan Eagles
      2. YARN-2513.v4.patch
        9 kB
        Jonathan Eagles
      3. YARN-2513.v5.patch
        14 kB
        Jonathan Eagles
      4. YARN-2513-v1.patch
        6 kB
        Jonathan Eagles
      5. YARN-2513-v2.patch
        6 kB
        Jonathan Eagles

        Issue Links

          Activity

          Hide
          jeagles Jonathan Eagles added a comment -

          Vinod Kumar Vavilapalli, Hitesh Shah, Zhijie Shen, I have posted a patch that will simply allow the timeline server to host generic UI that still pass through the web filters of hadoop. Please give some feedback.

              <property>
                  <name>yarn.timeline-service.ui-names</name>
                  <value>tez</value>
              </property>
              <property>
                  <name>yarn.timeline-service.ui-on-disk-path.tez</name>
                  <value>/Users/jeagles/hadoop/tez-ui</value>
              </property>
              <property>
                  <name>yarn.timeline-service.ui-web-path.tez</name>
                  <value>/tez-ui-v1.0</value>
              </property>
          
          Show
          jeagles Jonathan Eagles added a comment - Vinod Kumar Vavilapalli , Hitesh Shah , Zhijie Shen , I have posted a patch that will simply allow the timeline server to host generic UI that still pass through the web filters of hadoop. Please give some feedback. <property> <name>yarn.timeline-service.ui-names</name> <value>tez</value> </property> <property> <name>yarn.timeline-service.ui-on-disk-path.tez</name> <value>/Users/jeagles/hadoop/tez-ui</value> </property> <property> <name>yarn.timeline-service.ui-web-path.tez</name> <value>/tez-ui-v1.0</value> </property>
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12668429/YARN-2513-v1.patch
          against trunk revision 98588cf.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/4952//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4952//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12668429/YARN-2513-v1.patch against trunk revision 98588cf. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/4952//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4952//console This message is automatically generated.
          Hide
          zjshen Zhijie Shen added a comment -

          Jonathan Eagles, thanks for working on this feature, which seems to be interesting. The benefit I can see with integrating the per-framework UI widget into the timeline server is to relieve the framework from deploying its own web server, and to prevent additional data move from the timeline server to the framework web server, and finally to the user.

          To enable the web plugin, so far I can think of the following things that it's good to address.

          1. Security: there're three potential issues. a) Do we want to use Hadoop http authentication to protect the pre-framework web plugin? b) We don't page-level or url matching access authorization, such that, for example, UserA can only access the web pages of it's authorized web plugin. c) Currently, everything inside timeline server belongs to YARN, such that we don't limit its access to the resources internally. Web plugin is hosted by the timeline server on behalf of the framework, and it should only have the access to the resources granted to it. For example, Tez webUI should only have access to the tez metrics in the timeline store.

          2. Isolation: for some reason, the web plugin is going to crash, we should make sure it is not going to affect other components in the timeline server and other web plugins. Moreover, if multiple web plugins are hosted in the timeline server, we need to take care of their competing for the web server resources, preventing starvation.

          3. Scalability: now everything is hosted in a single web server container. Hosting framework UIs will drive more traffic to the timeline server. We may want to scale up web server instances to handle users' requests. In addition, it's good think of whether we want distribute the workload according the functions: some of them serves raw REST APIs, others serve web UI of framework 1, 2, 3 and the remaining ones serve web UI of framework 4, 5, 6.

          Show
          zjshen Zhijie Shen added a comment - Jonathan Eagles , thanks for working on this feature, which seems to be interesting. The benefit I can see with integrating the per-framework UI widget into the timeline server is to relieve the framework from deploying its own web server, and to prevent additional data move from the timeline server to the framework web server, and finally to the user. To enable the web plugin, so far I can think of the following things that it's good to address. 1. Security: there're three potential issues. a) Do we want to use Hadoop http authentication to protect the pre-framework web plugin? b) We don't page-level or url matching access authorization, such that, for example, UserA can only access the web pages of it's authorized web plugin. c) Currently, everything inside timeline server belongs to YARN, such that we don't limit its access to the resources internally. Web plugin is hosted by the timeline server on behalf of the framework, and it should only have the access to the resources granted to it. For example, Tez webUI should only have access to the tez metrics in the timeline store. 2. Isolation: for some reason, the web plugin is going to crash, we should make sure it is not going to affect other components in the timeline server and other web plugins. Moreover, if multiple web plugins are hosted in the timeline server, we need to take care of their competing for the web server resources, preventing starvation. 3. Scalability: now everything is hosted in a single web server container. Hosting framework UIs will drive more traffic to the timeline server. We may want to scale up web server instances to handle users' requests. In addition, it's good think of whether we want distribute the workload according the functions: some of them serves raw REST APIs, others serve web UI of framework 1, 2, 3 and the remaining ones serve web UI of framework 4, 5, 6.
          Hide
          jeagles Jonathan Eagles added a comment -

          Refreshing the patch.

          Show
          jeagles Jonathan Eagles added a comment - Refreshing the patch.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12681180/YARN-2513-v2.patch
          against trunk revision d7150a1.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          -1 eclipse:eclipse. The patch failed to build with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5832//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5832//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12681180/YARN-2513-v2.patch against trunk revision d7150a1. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. -1 eclipse:eclipse . The patch failed to build with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5832//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5832//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12681180/YARN-2513-v2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / f1a152c
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7638/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12681180/YARN-2513-v2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f1a152c Console output https://builds.apache.org/job/PreCommit-YARN-Build/7638/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12681180/YARN-2513-v2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / f1a152c
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7647/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12681180/YARN-2513-v2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f1a152c Console output https://builds.apache.org/job/PreCommit-YARN-Build/7647/console This message was automatically generated.
          Hide
          zjshen Zhijie Shen added a comment -

          Jonathan Eagles, do we still want to have this for ATS v1?

          Show
          zjshen Zhijie Shen added a comment - Jonathan Eagles , do we still want to have this for ATS v1?
          Hide
          jeagles Jonathan Eagles added a comment -

          Zhijie Shen, this is the patch we've been running with for the past 5 months. Essentially, this adds all the filters (minus guice) including security. If you think there can be some value to others to have this as well we can proceed to get this in. At the very least we should make a go/no decision on this patch. I've posted the latest working patch for closure on this issue. Ultimately, I've been using this to deploy the Tez UI until another reliable hosting solution presents itself.

          Show
          jeagles Jonathan Eagles added a comment - Zhijie Shen , this is the patch we've been running with for the past 5 months. Essentially, this adds all the filters (minus guice) including security. If you think there can be some value to others to have this as well we can proceed to get this in. At the very least we should make a go/no decision on this patch. I've posted the latest working patch for closure on this issue. Ultimately, I've been using this to deploy the Tez UI until another reliable hosting solution presents itself.
          Hide
          hitesh Hitesh Shah added a comment -

          +1 to making this available for ATS v1. Would be useful in various deployments .

          Show
          hitesh Hitesh Shah added a comment - +1 to making this available for ATS v1. Would be useful in various deployments .
          Hide
          zjshen Zhijie Shen added a comment -

          As it's valuable to some existing ATS use case, let's try to get the patch in and target 2.8.

          Jonathan Eagles, three comments about the patch:

          1. Shall we add "yarn.timeline-service.ui-names" to yarn-default.xml too? Like "yarn.nodemanager.aux-services"?

          2. Can we add some text in TimelineServer.md to document the configs and introduce how to install framework UIs.

          3. Can we add a test case to validate and showcase that ATS can load a framework UIs (e.g., a single helloworld.html)?

          Show
          zjshen Zhijie Shen added a comment - As it's valuable to some existing ATS use case, let's try to get the patch in and target 2.8. Jonathan Eagles , three comments about the patch: 1. Shall we add "yarn.timeline-service.ui-names" to yarn-default.xml too? Like "yarn.nodemanager.aux-services"? 2. Can we add some text in TimelineServer.md to document the configs and introduce how to install framework UIs. 3. Can we add a test case to validate and showcase that ATS can load a framework UIs (e.g., a single helloworld.html)?
          Hide
          hitesh Hitesh Shah added a comment -

          I am not sure if multiple uis work.

          Tried the following configs:

            <property>
              <name>yarn.timeline-service.ui-names</name>
              <value>tezui,tezui2</value>
            </property>
          
            <property>
              <name>yarn.timeline-service.ui-web-path.tezui</name>
              <value>/tezui</value>
            </property>
          
            <property>
              <name>yarn.timeline-service.ui-on-disk-path.tezui</name>
              <value>/install//tez/ui/</value>
            </property>
          
          
            <property>
              <name>yarn.timeline-service.ui-web-path.tezui2</name>
              <value>/tezui2</value>
            </property>
          
            <property>
              <name>yarn.timeline-service.ui-on-disk-path.tezui2</name>
              <value>/install/tez/tez-ui-0.8.0-SNAPSHOT.war</value>
            </property>
          

          Logs:

          2015-07-22 22:43:09,643 ERROR applicationhistoryservice.ApplicationHistoryServer - AHSWebApp failed to start.
          java.lang.NullPointerException
                  at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.startWebApp(ApplicationHistoryServer.java:295)
                  at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:114)
                  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
                  at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:162)
                  at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:171)
          2015-07-22 22:43:09,644 INFO  service.AbstractService - Service org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: AHSWebApp failed to start.
          org.apache.hadoop.yarn.exceptions.YarnRuntimeException: AHSWebApp failed to start.
                  at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.startWebApp(ApplicationHistoryServer.java:305)
                  at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:114)
                  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
                  at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:162)
                  at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:171)
          Caused by: java.lang.NullPointerException
                  at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.startWebApp(ApplicationHistoryServer.java:295)
                  ... 4 more
          
          Show
          hitesh Hitesh Shah added a comment - I am not sure if multiple uis work. Tried the following configs: <property> <name>yarn.timeline-service.ui-names</name> <value>tezui,tezui2</value> </property> <property> <name>yarn.timeline-service.ui-web-path.tezui</name> <value>/tezui</value> </property> <property> <name>yarn.timeline-service.ui-on-disk-path.tezui</name> <value>/install //tez/ui/</value> </property> <property> <name>yarn.timeline-service.ui-web-path.tezui2</name> <value>/tezui2</value> </property> <property> <name>yarn.timeline-service.ui-on-disk-path.tezui2</name> <value>/install/tez/tez-ui-0.8.0-SNAPSHOT.war</value> </property> Logs: 2015-07-22 22:43:09,643 ERROR applicationhistoryservice.ApplicationHistoryServer - AHSWebApp failed to start. java.lang.NullPointerException at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.startWebApp(ApplicationHistoryServer.java:295) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:114) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:162) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:171) 2015-07-22 22:43:09,644 INFO service.AbstractService - Service org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: AHSWebApp failed to start. org.apache.hadoop.yarn.exceptions.YarnRuntimeException: AHSWebApp failed to start. at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.startWebApp(ApplicationHistoryServer.java:305) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:114) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:162) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:171) Caused by: java.lang.NullPointerException at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.startWebApp(ApplicationHistoryServer.java:295) ... 4 more
          Hide
          xgong Xuan Gong added a comment -

          Jonathan Eagles Do we have any updates for this ?

          Show
          xgong Xuan Gong added a comment - Jonathan Eagles Do we have any updates for this ?
          Hide
          jeagles Jonathan Eagles added a comment -

          I'll need to have a look at this to see why two UI's don't work.

          Show
          jeagles Jonathan Eagles added a comment - I'll need to have a look at this to see why two UI's don't work.
          Hide
          jeagles Jonathan Eagles added a comment -

          Hitesh Shah, fixed the NPE on multiple UIs hosted. It looks like the filterHolders were being invalidated after the first UI filter was defined.

          Zhijie Shen, I added yarn-default.xml and TimelineServer.md changes. I didn't get a chance to add the test you suggest.

          Xuan Gong, I have updated the patch to help get this going again.

          Show
          jeagles Jonathan Eagles added a comment - Hitesh Shah , fixed the NPE on multiple UIs hosted. It looks like the filterHolders were being invalidated after the first UI filter was defined. Zhijie Shen , I added yarn-default.xml and TimelineServer.md changes. I didn't get a chance to add the test you suggest. Xuan Gong , I have updated the patch to help get this going again.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 23m 35s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 0s There were no new javac warning messages.
          +1 javadoc 10m 5s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 site 3m 7s Site still builds.
          -1 checkstyle 1m 46s The applied patch generated 6 new checkstyle issues (total was 211, now 216).
          -1 checkstyle 2m 14s The applied patch generated 1 new checkstyle issues (total was 38, now 38).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 4m 5s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 59s Tests passed in hadoop-yarn-common.
          +1 yarn tests 3m 15s Tests passed in hadoop-yarn-server-applicationhistoryservice.
              59m 19s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12755650/YARN-2513.v4.patch
          Optional Tests javadoc javac unit findbugs checkstyle site
          git revision trunk / 332b520
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9113/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt https://builds.apache.org/job/PreCommit-YARN-Build/9113/artifact/patchprocess/diffcheckstylehadoop-yarn-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9113/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9113/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9113/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9113/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9113/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 23m 35s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 0s There were no new javac warning messages. +1 javadoc 10m 5s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 site 3m 7s Site still builds. -1 checkstyle 1m 46s The applied patch generated 6 new checkstyle issues (total was 211, now 216). -1 checkstyle 2m 14s The applied patch generated 1 new checkstyle issues (total was 38, now 38). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 5s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 59s Tests passed in hadoop-yarn-common. +1 yarn tests 3m 15s Tests passed in hadoop-yarn-server-applicationhistoryservice.     59m 19s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12755650/YARN-2513.v4.patch Optional Tests javadoc javac unit findbugs checkstyle site git revision trunk / 332b520 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9113/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt https://builds.apache.org/job/PreCommit-YARN-Build/9113/artifact/patchprocess/diffcheckstylehadoop-yarn-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9113/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9113/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9113/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9113/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9113/console This message was automatically generated.
          Hide
          jeagles Jonathan Eagles added a comment -

          Fixing checkstyle issues and trying to implement a unit test as requested.

          Show
          jeagles Jonathan Eagles added a comment - Fixing checkstyle issues and trying to implement a unit test as requested.
          Hide
          jeagles Jonathan Eagles added a comment -

          Fixing (hopefully) the checkstyle warnings and adding a unit test to show functionality.

          Show
          jeagles Jonathan Eagles added a comment - Fixing (hopefully) the checkstyle warnings and adding a unit test to show functionality.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 21m 45s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 3s There were no new javac warning messages.
          +1 javadoc 10m 6s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 site 3m 2s Site still builds.
          -1 checkstyle 1m 38s The applied patch generated 1 new checkstyle issues (total was 211, now 211).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 4m 5s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common.
          +1 yarn tests 3m 11s Tests passed in hadoop-yarn-server-applicationhistoryservice.
              57m 17s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12755807/YARN-2513.v5.patch
          Optional Tests javadoc javac unit findbugs checkstyle site
          git revision trunk / 6955771
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9131/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9131/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9131/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9131/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9131/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9131/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 21m 45s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 3s There were no new javac warning messages. +1 javadoc 10m 6s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 site 3m 2s Site still builds. -1 checkstyle 1m 38s The applied patch generated 1 new checkstyle issues (total was 211, now 211). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 5s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common. +1 yarn tests 3m 11s Tests passed in hadoop-yarn-server-applicationhistoryservice.     57m 17s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12755807/YARN-2513.v5.patch Optional Tests javadoc javac unit findbugs checkstyle site git revision trunk / 6955771 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9131/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9131/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9131/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9131/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9131/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9131/console This message was automatically generated.
          Hide
          jeagles Jonathan Eagles added a comment -

          Checkstyle issue is not related to patch.

          ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java:1: File length is 2,153 lines (max allowed is 2,000).

          Hitesh Shah, Xuan Gong, Zhijie Shen, can you have another look at this patch when you get a chance?

          Show
          jeagles Jonathan Eagles added a comment - Checkstyle issue is not related to patch. ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java:1: File length is 2,153 lines (max allowed is 2,000). Hitesh Shah , Xuan Gong , Zhijie Shen , can you have another look at this patch when you get a chance?
          Hide
          jeagles Jonathan Eagles added a comment -

          Xuan Gong, Zhijie Shen, will you have time to review this patch?

          Show
          jeagles Jonathan Eagles added a comment - Xuan Gong , Zhijie Shen , will you have time to review this patch?
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Dropping 2.6.1-candidate label, 2.6.1 is out now.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Dropping 2.6.1-candidate label, 2.6.1 is out now.
          Hide
          hitesh Hitesh Shah added a comment -

          Tested the latest patch with multiple UIs being hosted. Works fine now.

          Show
          hitesh Hitesh Shah added a comment - Tested the latest patch with multiple UIs being hosted. Works fine now.
          Hide
          hitesh Hitesh Shah added a comment -

          +1

          Show
          hitesh Hitesh Shah added a comment - +1
          Hide
          jeagles Jonathan Eagles added a comment -

          Thanks, everybody. Committed this to trunk branch-2 and branch-2.7

          Show
          jeagles Jonathan Eagles added a comment - Thanks, everybody. Committed this to trunk branch-2 and branch-2.7
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8680 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8680/)
          YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8680 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8680/ ) YARN-2513 . Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1300 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1300/)
          YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1300 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1300/ ) YARN-2513 . Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #579 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/579/)
          YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #579 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/579/ ) YARN-2513 . Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2512 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2512/)
          YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2512 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2512/ ) YARN-2513 . Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #565 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/565/)
          YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #565 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/565/ ) YARN-2513 . Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #523 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/523/)
          YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #523 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/523/ ) YARN-2513 . Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2460 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2460/)
          YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2460 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2460/ ) YARN-2513 . Host framework UIs in YARN for use with the ATS (jeagles) (jeagles: rev 25f8f801d15e3d9f27f4a2a198262407203e14a5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          Hide
          hitesh Hitesh Shah added a comment -

          Yesha Vora and I are trying to use this on a secure cluster. I can kinit and make a curl call to
          "/ws/v1/timeline/TEZ_DAG_ID?limit=1" and it works correctly but when trying to make a curl call to the hosted UI, it fails.

          I see that KerberosAuthenticationHandler.java:init(214) is being invoked twice.

          The error being thrown is:

          2015-11-03 05:01:45,864 WARN  server.AuthenticationFilter (AuthenticationFilter.java:doFilter(551)) - Authentication exception: GSSException: Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34))
          org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34))
                  at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.authenticate(KerberosAuthenticationHandler.java:399)
                  at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:347)
                  at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507)
                  at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
                  at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1225)
                  at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
                  at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
                  at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
                  at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
                  at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
                  at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
                  at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
                  at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
                  at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
                  at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
          

          Jonathan Eagles Vinod Kumar Vavilapalli Steve Loughran Any suggestions?

          Show
          hitesh Hitesh Shah added a comment - Yesha Vora and I are trying to use this on a secure cluster. I can kinit and make a curl call to "/ws/v1/timeline/TEZ_DAG_ID?limit=1" and it works correctly but when trying to make a curl call to the hosted UI, it fails. I see that KerberosAuthenticationHandler.java:init(214) is being invoked twice. The error being thrown is: 2015-11-03 05:01:45,864 WARN server.AuthenticationFilter (AuthenticationFilter.java:doFilter(551)) - Authentication exception: GSSException: Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34)) org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34)) at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.authenticate(KerberosAuthenticationHandler.java:399) at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:347) at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1225) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) Jonathan Eagles Vinod Kumar Vavilapalli Steve Loughran Any suggestions?
          Hide
          hitesh Hitesh Shah added a comment -

          The below patch seems to work but I am not sure what else I may be breaking:

          --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/Applicatio
          +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/Applicatio
          @@ -305,16 +305,16 @@ private void startWebApp() {
                    WebAppContext uiWebAppContext = new WebAppContext();
                    uiWebAppContext.setContextPath(webPath);
                    uiWebAppContext.setWar(onDiskPath);
          -         final String[] ALL_URLS = { "/*" };
          -         FilterHolder[] filterHolders =
          -           webAppContext.getServletHandler().getFilters();
          -         for (FilterHolder filterHolder: filterHolders) {
          -           if (!"guice".equals(filterHolder.getName())) {
          -             HttpServer2.defineFilter(uiWebAppContext, filterHolder.getName(),
          -                 filterHolder.getClassName(), filterHolder.getInitParameters(),
          -                 ALL_URLS);
          -           }
          -         }
          +         //final String[] ALL_URLS = { "/*" };
          +         //FilterHolder[] filterHolders =
          +         //  webAppContext.getServletHandler().getFilters();
          +         //for (FilterHolder filterHolder: filterHolders) {
          +         //  if (!"guice".equals(filterHolder.getName())) {
          +         //    HttpServer2.defineFilter(uiWebAppContext, filterHolder.getName(),
          +         //        filterHolder.getClassName(), filterHolder.getInitParameters(),
          +         //        ALL_URLS);
          +         //  }
          +         //}
          
          Show
          hitesh Hitesh Shah added a comment - The below patch seems to work but I am not sure what else I may be breaking: --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/Applicatio +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/Applicatio @@ -305,16 +305,16 @@ private void startWebApp() { WebAppContext uiWebAppContext = new WebAppContext(); uiWebAppContext.setContextPath(webPath); uiWebAppContext.setWar(onDiskPath); - final String [] ALL_URLS = { "/*" }; - FilterHolder[] filterHolders = - webAppContext.getServletHandler().getFilters(); - for (FilterHolder filterHolder: filterHolders) { - if (! "guice" .equals(filterHolder.getName())) { - HttpServer2.defineFilter(uiWebAppContext, filterHolder.getName(), - filterHolder.getClassName(), filterHolder.getInitParameters(), - ALL_URLS); - } - } + // final String [] ALL_URLS = { "/*" }; + //FilterHolder[] filterHolders = + // webAppContext.getServletHandler().getFilters(); + // for (FilterHolder filterHolder: filterHolders) { + // if (! "guice" .equals(filterHolder.getName())) { + // HttpServer2.defineFilter(uiWebAppContext, filterHolder.getName(), + // filterHolder.getClassName(), filterHolder.getInitParameters(), + // ALL_URLS); + // } + //}

            People

            • Assignee:
              jeagles Jonathan Eagles
              Reporter:
              jeagles Jonathan Eagles
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development