Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5076

YARN web interfaces lack XFS protection

    Details

    • Hadoop Flags:
      Reviewed

      Description

      There are web interfaces in YARN that do not provide protection against cross frame scripting (https://www.owasp.org/index.php/Clickjacking_Defense_Cheat_Sheet). HADOOP-13008 provides a common filter for addressing this vulnerability, so this filter should be integrated into the YARN web interfaces.

      1. YARN-5076.002.patch
        21 kB
        Jonathan Maron
      2. YARN-5076.003.patch
        20 kB
        Jonathan Maron
      3. YARN-5076.004.patch
        20 kB
        Jonathan Maron

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #9826 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9826/)
        YARN-5076. YARN web interfaces lack XFS protection. Contributed by (junping_du: rev 22fcd819f0c445be661e644ed67221f867013af8)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/webapp/TestRMWithXFSFilter.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/WebServer.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9826 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9826/ ) YARN-5076 . YARN web interfaces lack XFS protection. Contributed by (junping_du: rev 22fcd819f0c445be661e644ed67221f867013af8) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/webapp/TestRMWithXFSFilter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/WebServer.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        djp Junping Du added a comment -

        I have commit the patch to trunk and branch-2. Thanks Jonathan Maron for the patch and Varun Vasudev for review!

        Show
        djp Junping Du added a comment - I have commit the patch to trunk and branch-2. Thanks Jonathan Maron for the patch and Varun Vasudev for review!
        Hide
        djp Junping Du added a comment -

        The other check-style issue is just a minor whitespace issue. Will fix it in commit.
        +1 on 004 patch. Will commit it shortly.

        Show
        djp Junping Du added a comment - The other check-style issue is just a minor whitespace issue. Will fix it in commit. +1 on 004 patch. Will commit it shortly.
        Hide
        djp Junping Du added a comment -

        That's right. Just check again that it sounds like we were not adding package-info for all test classes' package. Ok. Let's keep it is right now.

        Show
        djp Junping Du added a comment - That's right. Just check again that it sounds like we were not adding package-info for all test classes' package. Ok. Let's keep it is right now.
        Hide
        jmaron Jonathan Maron added a comment -

        Does a package-info.java file need to exist for org.apache.hadoop.yarn.server.resourcemanager.webapp? I don't see one at the moment.

        HW10386:hadoop jmaron$ find . -name package-info.java | grep resource
        ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/package-info.java
        
        Show
        jmaron Jonathan Maron added a comment - Does a package-info.java file need to exist for org.apache.hadoop.yarn.server.resourcemanager.webapp? I don't see one at the moment. HW10386:hadoop jmaron$ find . -name package-info.java | grep resource ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/package-info.java
        Hide
        djp Junping Du added a comment -

        I think so - in this way, you have to also update the package name to "org.apache.hadoop.yarn.server.resourcemanager.webapp;" after the movement. Or you have to add a package-info.java with specifying new package name in the previous directory. I am fine with both ways but prefer the previous way.

        Show
        djp Junping Du added a comment - I think so - in this way, you have to also update the package name to "org.apache.hadoop.yarn.server.resourcemanager.webapp;" after the movement. Or you have to add a package-info.java with specifying new package name in the previous directory. I am fine with both ways but prefer the previous way.
        Hide
        jmaron Jonathan Maron added a comment -

        Will that address this problem? I have no idea why this would be required simply to accommodate a test class.

        Show
        jmaron Jonathan Maron added a comment - Will that address this problem? I have no idea why this would be required simply to accommodate a test class.
        Hide
        djp Junping Du added a comment -

        The unit test failure is not related and we have other JIRA to track this.
        However, we need to fix the checkstyle issue, especially the "missing package-info.java file": I think it could be better to move the test class from ".../src/test/java/org/apache/hadoop/yarn/webapp/" to ".../src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/" given we don't test other daemon's webapp.

        Show
        djp Junping Du added a comment - The unit test failure is not related and we have other JIRA to track this. However, we need to fix the checkstyle issue, especially the "missing package-info.java file": I think it could be better to move the test class from ".../src/test/java/org/apache/hadoop/yarn/webapp/" to ".../src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/" given we don't test other daemon's webapp.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 7m 25s trunk passed
        +1 compile 6m 40s trunk passed
        +1 checkstyle 1m 24s trunk passed
        +1 mvnsite 3m 54s trunk passed
        +1 mvneclipse 1m 44s trunk passed
        +1 findbugs 7m 39s trunk passed
        +1 javadoc 3m 41s trunk passed
        0 mvndep 0m 18s Maven dependency ordering for patch
        +1 mvninstall 3m 43s the patch passed
        +1 compile 8m 58s the patch passed
        +1 javac 8m 58s the patch passed
        -1 checkstyle 1m 40s root: patch generated 2 new + 432 unchanged - 0 fixed = 434 total (was 432)
        +1 mvnsite 4m 6s the patch passed
        +1 mvneclipse 1m 38s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 7m 12s the patch passed
        +1 javadoc 3m 11s the patch passed
        +1 unit 0m 27s hadoop-yarn-api in the patch passed.
        +1 unit 2m 8s hadoop-yarn-common in the patch passed.
        +1 unit 11m 13s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 2m 58s hadoop-yarn-server-applicationhistoryservice in the patch passed.
        -1 unit 29m 24s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 1m 58s hadoop-mapreduce-client-core in the patch passed.
        +1 unit 0m 41s hadoop-mapreduce-client-common in the patch passed.
        +1 unit 5m 59s hadoop-mapreduce-client-hs in the patch passed.
        +1 asflicense 0m 20s Patch does not generate ASF License warnings.
        120m 32s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804932/YARN-5076.004.patch
        JIRA Issue YARN-5076
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 2547e9f39d1f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 22ff9e6
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11553/artifact/patchprocess/diff-checkstyle-root.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11553/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11553/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11553/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11553/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 7m 25s trunk passed +1 compile 6m 40s trunk passed +1 checkstyle 1m 24s trunk passed +1 mvnsite 3m 54s trunk passed +1 mvneclipse 1m 44s trunk passed +1 findbugs 7m 39s trunk passed +1 javadoc 3m 41s trunk passed 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 3m 43s the patch passed +1 compile 8m 58s the patch passed +1 javac 8m 58s the patch passed -1 checkstyle 1m 40s root: patch generated 2 new + 432 unchanged - 0 fixed = 434 total (was 432) +1 mvnsite 4m 6s the patch passed +1 mvneclipse 1m 38s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 7m 12s the patch passed +1 javadoc 3m 11s the patch passed +1 unit 0m 27s hadoop-yarn-api in the patch passed. +1 unit 2m 8s hadoop-yarn-common in the patch passed. +1 unit 11m 13s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 2m 58s hadoop-yarn-server-applicationhistoryservice in the patch passed. -1 unit 29m 24s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 1m 58s hadoop-mapreduce-client-core in the patch passed. +1 unit 0m 41s hadoop-mapreduce-client-common in the patch passed. +1 unit 5m 59s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 120m 32s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804932/YARN-5076.004.patch JIRA Issue YARN-5076 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 2547e9f39d1f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 22ff9e6 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11553/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11553/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11553/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11553/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11553/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        djp Junping Du added a comment -

        No worry. I already kicked it off: https://builds.apache.org/job/PreCommit-YARN-Build/11553/
        Let's wait for result.

        Show
        djp Junping Du added a comment - No worry. I already kicked it off: https://builds.apache.org/job/PreCommit-YARN-Build/11553/ Let's wait for result.
        Hide
        jmaron Jonathan Maron added a comment -

        You kicked it off or do you want me to? I have been unsuccessful in the past...

        Show
        jmaron Jonathan Maron added a comment - You kicked it off or do you want me to? I have been unsuccessful in the past...
        Hide
        djp Junping Du added a comment -

        It looks like Mr. Jenkins is on strike for some reason. Kick off it manually.

        Show
        djp Junping Du added a comment - It looks like Mr. Jenkins is on strike for some reason. Kick off it manually.
        Hide
        djp Junping Du added a comment -

        Got it. If we don't set any CUSTOM_HEADER_PARAM, XFrameOptionsFilter will set default option as DENY.
        +1 on latest patch pending on Jenkins' report.

        Show
        djp Junping Du added a comment - Got it. If we don't set any CUSTOM_HEADER_PARAM, XFrameOptionsFilter will set default option as DENY. +1 on latest patch pending on Jenkins' report.
        Hide
        jmaron Jonathan Maron added a comment -

        The test actually verified that a value set for the property is returned as a header, so the value isn't as important as the fact that the values are the same.

        Show
        jmaron Jonathan Maron added a comment - The test actually verified that a value set for the property is returned as a header, so the value isn't as important as the fact that the values are the same.
        Hide
        djp Junping Du added a comment -

        Thanks Jon. v4 patch get very closed.
        Just one issue: testDefaultBehavior() still test against DENY but we already change the default option to SAMEORIGIN. However, when I run the test, it can finish successfully. Do I miss anything here?

        Show
        djp Junping Du added a comment - Thanks Jon. v4 patch get very closed. Just one issue: testDefaultBehavior() still test against DENY but we already change the default option to SAMEORIGIN. However, when I run the test, it can finish successfully. Do I miss anything here?
        Hide
        jmaron Jonathan Maron added a comment -

        Thanks - missed that. Will upload new patch.

        Show
        jmaron Jonathan Maron added a comment - Thanks - missed that. Will upload new patch.
        Hide
        djp Junping Du added a comment -

        Thanks Jonathan Maron for updating the patch.
        In yarn-default.xml, we still put configuration "yarn.nodemanager.webapp.xfs-filter.xframe-options-enabled" and "yarn.timeline-service.webapp.xfs-filter.xframe-options-enabled" there. We should remove them.
        Other looks fine to me.

        Show
        djp Junping Du added a comment - Thanks Jonathan Maron for updating the patch. In yarn-default.xml, we still put configuration "yarn.nodemanager.webapp.xfs-filter.xframe-options-enabled" and "yarn.timeline-service.webapp.xfs-filter.xframe-options-enabled" there. We should remove them. Other looks fine to me.
        Hide
        jmaron Jonathan Maron added a comment -

        Varun Vasudev - please review and see if this addresses the changes requested. Thanks!

        Show
        jmaron Jonathan Maron added a comment - Varun Vasudev - please review and see if this addresses the changes requested. Thanks!
        Hide
        djp Junping Du added a comment -

        Yes. Varun's list above is exactly what I prefer. Please go ahead to update. Also, SAMEORIGIN as default make more sense also. Thanks!

        Show
        djp Junping Du added a comment - Yes. Varun's list above is exactly what I prefer. Please go ahead to update. Also, SAMEORIGIN as default make more sense also. Thanks!
        Hide
        jmaron Jonathan Maron added a comment -

        Junping Du - if you can confirm that the settings Varun Vasudev lists above are a reflection of the way you'd like to see the configuration I can make the change - I have no real strong objection. I can look into making the default SAMEORIGIN.

        Show
        jmaron Jonathan Maron added a comment - Junping Du - if you can confirm that the settings Varun Vasudev lists above are a reflection of the way you'd like to see the configuration I can make the change - I have no real strong objection. I can look into making the default SAMEORIGIN.
        Hide
        vvasudev Varun Vasudev added a comment -

        Ah I think I misunderstood what you were saying. If I understand you correctly, you want the following the configs -

        1. yarn.webapp.xfs-filter.enabled
        2. yarn.resourcemanager.webapp.xframe-options
        3. yarn.nodemanager.webapp.xframe-options
        4. yarn.timeline-service.webapp.xframe-options

        Correct?

        Show
        vvasudev Varun Vasudev added a comment - Ah I think I misunderstood what you were saying. If I understand you correctly, you want the following the configs - yarn.webapp.xfs-filter.enabled yarn.resourcemanager.webapp.xframe-options yarn.nodemanager.webapp.xframe-options yarn.timeline-service.webapp.xframe-options Correct?
        Hide
        djp Junping Du added a comment -

        Like I mentioned above - embedding the RM/ATS ui in a frame but blocking the NM ui is a pretty reasonable scenario.

        Agree. I think we can achieve this by set RM/ATS's option to SAMEORIGIN but keep NM as DENY. Isn't it?

        Adding a YARN level config which can then be overridden by a RM level config down the line will make things more confusing.

        There is no overridden here. A YARN level configuration is just to enable/disable XFS protection feature. The sub options to address different daemons' requirement if XFS protection is enabled. Do I miss any cases here?

        It's the other way round that's the problem in my opinion - with one config parameter - you force the users to open all web ui's or no web ui's.

        Not really. The one config parameter here is just to mark YARN web ui are open or restricted (in different levels/options). Is there really a case we want some YARN web ui pure open to frame when other is protected? Instead, adding configurable ALLOW-FROM make more sense to me.

        Show
        djp Junping Du added a comment - Like I mentioned above - embedding the RM/ATS ui in a frame but blocking the NM ui is a pretty reasonable scenario. Agree. I think we can achieve this by set RM/ATS's option to SAMEORIGIN but keep NM as DENY. Isn't it? Adding a YARN level config which can then be overridden by a RM level config down the line will make things more confusing. There is no overridden here. A YARN level configuration is just to enable/disable XFS protection feature. The sub options to address different daemons' requirement if XFS protection is enabled. Do I miss any cases here? It's the other way round that's the problem in my opinion - with one config parameter - you force the users to open all web ui's or no web ui's. Not really. The one config parameter here is just to mark YARN web ui are open or restricted (in different levels/options). Is there really a case we want some YARN web ui pure open to frame when other is protected? Instead, adding configurable ALLOW-FROM make more sense to me.
        Hide
        vvasudev Varun Vasudev added a comment -

        Like I mentioned above - embedding the RM/ATS ui in a frame but blocking the NM ui is a pretty reasonable scenario. Adding a YARN level config which can then be overridden by a RM level config down the line will make things more confusing. It's the other way round that's the problem in my opinion - with one config parameter - you force the users to open all web ui's or no web ui's.

        Show
        vvasudev Varun Vasudev added a comment - Like I mentioned above - embedding the RM/ATS ui in a frame but blocking the NM ui is a pretty reasonable scenario. Adding a YARN level config which can then be overridden by a RM level config down the line will make things more confusing. It's the other way round that's the problem in my opinion - with one config parameter - you force the users to open all web ui's or no web ui's.
        Hide
        djp Junping Du added a comment -

        I believe we need the granular control that Jon has implemented.

        You mean granular enable/disable on different web app? Do we have solid case for now? If not, we can be more "granular" later. As I mentioned earlier, it always much easier to add some configurations than remove them.

        Show
        djp Junping Du added a comment - I believe we need the granular control that Jon has implemented. You mean granular enable/disable on different web app? Do we have solid case for now? If not, we can be more "granular" later. As I mentioned earlier, it always much easier to add some configurations than remove them.
        Hide
        vvasudev Varun Vasudev added a comment -

        I'm fine with changing the default to SAMEORIGIN. I believe we need the granular control that Jon has implemented.

        Show
        vvasudev Varun Vasudev added a comment - I'm fine with changing the default to SAMEORIGIN. I believe we need the granular control that Jon has implemented.
        Hide
        djp Junping Du added a comment -

        Granular control on different XFS options sounds reasonable. However, back to my original question - do we need to partially enable XFS protection? If not, at least we can merge all "xframe-options-enabled" configurations. I want to call loudly for our attention on adding new configuration carefully: once unnecessary configurations are added, it would be very hard to remove.
        In addition, mark XFS protection enabled with DENY option by default is a kind of behavior change - that could affect monitoring tools like Ambari. Shall we disable it by default?

        Show
        djp Junping Du added a comment - Granular control on different XFS options sounds reasonable. However, back to my original question - do we need to partially enable XFS protection? If not, at least we can merge all "xframe-options-enabled" configurations. I want to call loudly for our attention on adding new configuration carefully: once unnecessary configurations are added, it would be very hard to remove. In addition, mark XFS protection enabled with DENY option by default is a kind of behavior change - that could affect monitoring tools like Ambari. Shall we disable it by default?
        Hide
        jmaron Jonathan Maron added a comment -

        I agree with a Varun Vasudev - there are certainly use cases in which the granular control of this filter would be required.

        Show
        jmaron Jonathan Maron added a comment - I agree with a Varun Vasudev - there are certainly use cases in which the granular control of this filter would be required.
        Hide
        vvasudev Varun Vasudev added a comment -

        I think of a few cases around Ambari for example which may want to embed the RM UI but not the NM or ATS. In that case we want only the RM to have the XFS options set to SAMEORIGIN but the rest to DENY.

        Show
        vvasudev Varun Vasudev added a comment - I think of a few cases around Ambari for example which may want to embed the RM UI but not the NM or ATS. In that case we want only the RM to have the XFS options set to SAMEORIGIN but the rest to DENY.
        Hide
        djp Junping Du added a comment -

        Hi Jonathan Maron and Varun Vasudev, do we have very solid case to partially enable XFS protection (like enable RM, NM, but disable ATS)?
        If not, I would suggest to have a uniformed configuration "yarn.webapp.xfs-filter.xframe-options-enabled" and "yarn.webapp.xfs-filter.xframe-options" as we don't want too many redundant configurations in YARN/MAPREDUCE which is really a headache now.

        Show
        djp Junping Du added a comment - Hi Jonathan Maron and Varun Vasudev , do we have very solid case to partially enable XFS protection (like enable RM, NM, but disable ATS)? If not, I would suggest to have a uniformed configuration "yarn.webapp.xfs-filter.xframe-options-enabled" and "yarn.webapp.xfs-filter.xframe-options" as we don't want too many redundant configurations in YARN/MAPREDUCE which is really a headache now.
        Hide
        vvasudev Varun Vasudev added a comment -

        The resourcemanager test failures are known - it's due to an environment issue. The nodemanager and application history service test failures are new to me but are unrelated to this patch.

        +1 for the latest patch - I'll commit it tomorrow if no one objects.

        Show
        vvasudev Varun Vasudev added a comment - The resourcemanager test failures are known - it's due to an environment issue. The nodemanager and application history service test failures are new to me but are unrelated to this patch. +1 for the latest patch - I'll commit it tomorrow if no one objects.
        Hide
        jmaron Jonathan Maron added a comment -

        Varun Vasudev - the unit test failures above don't seem related to my changes. Is there anyway to find out if they are known issues?

        Show
        jmaron Jonathan Maron added a comment - Varun Vasudev - the unit test failures above don't seem related to my changes. Is there anyway to find out if they are known issues?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 8m 26s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 1m 20s Maven dependency ordering for branch
        +1 mvninstall 7m 27s trunk passed
        +1 compile 7m 10s trunk passed
        +1 checkstyle 1m 24s trunk passed
        +1 mvnsite 3m 19s trunk passed
        +1 mvneclipse 1m 32s trunk passed
        +1 findbugs 5m 39s trunk passed
        +1 javadoc 2m 52s trunk passed
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 2m 45s the patch passed
        +1 compile 6m 37s the patch passed
        +1 javac 6m 37s the patch passed
        +1 checkstyle 1m 23s the patch passed
        +1 mvnsite 3m 27s the patch passed
        +1 mvneclipse 1m 33s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 6m 53s the patch passed
        +1 javadoc 3m 16s the patch passed
        +1 unit 0m 26s hadoop-yarn-api in the patch passed.
        +1 unit 2m 18s hadoop-yarn-common in the patch passed.
        -1 unit 11m 18s hadoop-yarn-server-nodemanager in the patch failed.
        -1 unit 3m 2s hadoop-yarn-server-applicationhistoryservice in the patch failed.
        -1 unit 29m 48s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 2m 2s hadoop-mapreduce-client-core in the patch passed.
        +1 unit 0m 42s hadoop-mapreduce-client-common in the patch passed.
        +1 unit 5m 46s hadoop-mapreduce-client-hs in the patch passed.
        +1 asflicense 0m 22s Patch does not generate ASF License warnings.
        122m 32s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics
          hadoop.yarn.server.applicationhistoryservice.TestFileSystemApplicationHistoryStore
          hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestRMRestart
          hadoop.yarn.server.resourcemanager.TestAMAuthorization



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803772/YARN-5076.002.patch
        JIRA Issue YARN-5076
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 3fab534206a7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 34fddd1
        Default Java 1.8.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11502/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11502/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 8m 26s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 20s Maven dependency ordering for branch +1 mvninstall 7m 27s trunk passed +1 compile 7m 10s trunk passed +1 checkstyle 1m 24s trunk passed +1 mvnsite 3m 19s trunk passed +1 mvneclipse 1m 32s trunk passed +1 findbugs 5m 39s trunk passed +1 javadoc 2m 52s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 2m 45s the patch passed +1 compile 6m 37s the patch passed +1 javac 6m 37s the patch passed +1 checkstyle 1m 23s the patch passed +1 mvnsite 3m 27s the patch passed +1 mvneclipse 1m 33s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 6m 53s the patch passed +1 javadoc 3m 16s the patch passed +1 unit 0m 26s hadoop-yarn-api in the patch passed. +1 unit 2m 18s hadoop-yarn-common in the patch passed. -1 unit 11m 18s hadoop-yarn-server-nodemanager in the patch failed. -1 unit 3m 2s hadoop-yarn-server-applicationhistoryservice in the patch failed. -1 unit 29m 48s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 2m 2s hadoop-mapreduce-client-core in the patch passed. +1 unit 0m 42s hadoop-mapreduce-client-common in the patch passed. +1 unit 5m 46s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 122m 32s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics   hadoop.yarn.server.applicationhistoryservice.TestFileSystemApplicationHistoryStore   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803772/YARN-5076.002.patch JIRA Issue YARN-5076 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 3fab534206a7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 34fddd1 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt https://builds.apache.org/job/PreCommit-YARN-Build/11502/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11502/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11502/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jmaron Jonathan Maron added a comment -

        Attempting submission of patch 2 again.

        Show
        jmaron Jonathan Maron added a comment - Attempting submission of patch 2 again.
        Hide
        jmaron Jonathan Maron added a comment -

        Patch not picked up. Trying again.

        Show
        jmaron Jonathan Maron added a comment - Patch not picked up. Trying again.
        Hide
        jmaron Jonathan Maron added a comment -

        Submitting patch with NPE fix.

        Show
        jmaron Jonathan Maron added a comment - Submitting patch with NPE fix.
        Hide
        jmaron Jonathan Maron added a comment -

        Fix for NPE discovered through wider testing

        Show
        jmaron Jonathan Maron added a comment - Fix for NPE discovered through wider testing
        Hide
        jmaron Jonathan Maron added a comment -

        Resolving NPE

        Show
        jmaron Jonathan Maron added a comment - Resolving NPE
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 21s Maven dependency ordering for branch
        +1 mvninstall 6m 51s trunk passed
        +1 compile 6m 23s trunk passed with JDK v1.8.0_91
        +1 compile 6m 54s trunk passed with JDK v1.7.0_95
        +1 checkstyle 1m 33s trunk passed
        +1 mvnsite 3m 43s trunk passed
        +1 mvneclipse 1m 47s trunk passed
        -1 findbugs 1m 8s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in trunk has 1 extant Findbugs warnings.
        +1 javadoc 2m 56s trunk passed with JDK v1.8.0_91
        +1 javadoc 5m 40s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvninstall 3m 8s the patch passed
        +1 compile 6m 7s the patch passed with JDK v1.8.0_91
        +1 javac 6m 7s the patch passed
        +1 compile 6m 42s the patch passed with JDK v1.7.0_95
        +1 javac 6m 42s the patch passed
        -1 checkstyle 1m 29s root: patch generated 1 new + 435 unchanged - 0 fixed = 436 total (was 435)
        +1 mvnsite 3m 40s the patch passed
        +1 mvneclipse 1m 47s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 8m 54s the patch passed
        +1 javadoc 2m 59s the patch passed with JDK v1.8.0_91
        +1 javadoc 5m 43s the patch passed with JDK v1.7.0_95
        +1 unit 0m 22s hadoop-yarn-api in the patch passed with JDK v1.8.0_91.
        -1 unit 2m 4s hadoop-yarn-common in the patch failed with JDK v1.8.0_91.
        +1 unit 11m 8s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91.
        -1 unit 2m 58s hadoop-yarn-server-applicationhistoryservice in the patch failed with JDK v1.8.0_91.
        -1 unit 29m 34s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
        +1 unit 2m 0s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_91.
        +1 unit 0m 52s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_91.
        +1 unit 5m 42s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_91.
        +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.7.0_95.
        -1 unit 2m 28s hadoop-yarn-common in the patch failed with JDK v1.7.0_95.
        +1 unit 11m 39s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95.
        -1 unit 3m 13s hadoop-yarn-server-applicationhistoryservice in the patch failed with JDK v1.7.0_95.
        -1 unit 30m 48s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
        +1 unit 2m 20s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_95.
        +1 unit 0m 50s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_95.
        +1 unit 6m 7s hadoop-mapreduce-client-hs in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        199m 48s



        Reason Tests
        JDK v1.8.0_91 Failed junit tests hadoop.yarn.webapp.TestWebApp
          hadoop.yarn.server.applicationhistoryservice.TestFileSystemApplicationHistoryStore
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.webapp.TestWebApp
          hadoop.yarn.server.applicationhistoryservice.TestFileSystemApplicationHistoryStore
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
          hadoop.yarn.server.resourcemanager.TestAMAuthorization



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:cf2ee45
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803685/YARN-5076.001.patch
        JIRA Issue YARN-5076
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux d1eb2806a23a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0bbe01f
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/diff-checkstyle-root.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.8.0_91.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_91.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.7.0_95.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.7.0_95.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11429/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11429/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 21s Maven dependency ordering for branch +1 mvninstall 6m 51s trunk passed +1 compile 6m 23s trunk passed with JDK v1.8.0_91 +1 compile 6m 54s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 33s trunk passed +1 mvnsite 3m 43s trunk passed +1 mvneclipse 1m 47s trunk passed -1 findbugs 1m 8s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in trunk has 1 extant Findbugs warnings. +1 javadoc 2m 56s trunk passed with JDK v1.8.0_91 +1 javadoc 5m 40s trunk passed with JDK v1.7.0_95 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 3m 8s the patch passed +1 compile 6m 7s the patch passed with JDK v1.8.0_91 +1 javac 6m 7s the patch passed +1 compile 6m 42s the patch passed with JDK v1.7.0_95 +1 javac 6m 42s the patch passed -1 checkstyle 1m 29s root: patch generated 1 new + 435 unchanged - 0 fixed = 436 total (was 435) +1 mvnsite 3m 40s the patch passed +1 mvneclipse 1m 47s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 8m 54s the patch passed +1 javadoc 2m 59s the patch passed with JDK v1.8.0_91 +1 javadoc 5m 43s the patch passed with JDK v1.7.0_95 +1 unit 0m 22s hadoop-yarn-api in the patch passed with JDK v1.8.0_91. -1 unit 2m 4s hadoop-yarn-common in the patch failed with JDK v1.8.0_91. +1 unit 11m 8s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91. -1 unit 2m 58s hadoop-yarn-server-applicationhistoryservice in the patch failed with JDK v1.8.0_91. -1 unit 29m 34s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. +1 unit 2m 0s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_91. +1 unit 0m 52s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_91. +1 unit 5m 42s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_91. +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.7.0_95. -1 unit 2m 28s hadoop-yarn-common in the patch failed with JDK v1.7.0_95. +1 unit 11m 39s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95. -1 unit 3m 13s hadoop-yarn-server-applicationhistoryservice in the patch failed with JDK v1.7.0_95. -1 unit 30m 48s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 unit 2m 20s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_95. +1 unit 0m 50s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_95. +1 unit 6m 7s hadoop-mapreduce-client-hs in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 199m 48s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.yarn.webapp.TestWebApp   hadoop.yarn.server.applicationhistoryservice.TestFileSystemApplicationHistoryStore   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.webapp.TestWebApp   hadoop.yarn.server.applicationhistoryservice.TestFileSystemApplicationHistoryStore   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803685/YARN-5076.001.patch JIRA Issue YARN-5076 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux d1eb2806a23a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0bbe01f Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-warnings.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/11429/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11429/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11429/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jmaron Jonathan Maron added a comment -

        Initial patch

        Show
        jmaron Jonathan Maron added a comment - Initial patch
        Hide
        jmaron Jonathan Maron added a comment -

        This patch integrated the XFrame Options Filter from Hadoop Common. Some notes:

        • The filter is enabled by default. It can be disabled via 'xframe-options-enabled' properties (properties have been created for all web interfaces in this integration)
        • I've run checkstyle locally and there are no new style issues introduced by the code modified or added in this patch
        Show
        jmaron Jonathan Maron added a comment - This patch integrated the XFrame Options Filter from Hadoop Common. Some notes: The filter is enabled by default. It can be disabled via 'xframe-options-enabled' properties (properties have been created for all web interfaces in this integration) I've run checkstyle locally and there are no new style issues introduced by the code modified or added in this patch

          People

          • Assignee:
            jmaron Jonathan Maron
            Reporter:
            jmaron Jonathan Maron
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 48h
              48h
              Remaining:
              Remaining Estimate - 48h
              48h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development