Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4767

Network issues can cause persistent RM UI outage

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: webapp
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      If a network issue causes an AM web app to resolve the RM proxy's address to something other than what's listed in the allowed proxies list, the AmIpFilter will 302 redirect the RM proxy's request back to the RM proxy. The RM proxy will then consume all available handler threads connecting to itself over and over, resulting in an outage of the web UI.

      1. YARN-4767.001.patch
        5 kB
        Daniel Templeton
      2. YARN-4767.002.patch
        9 kB
        Daniel Templeton
      3. YARN-4767.003.patch
        22 kB
        Daniel Templeton
      4. YARN-4767.004.patch
        23 kB
        Daniel Templeton
      5. YARN-4767.005.patch
        23 kB
        Daniel Templeton
      6. YARN-4767.006.patch
        23 kB
        Daniel Templeton
      7. YARN-4767.007.patch
        24 kB
        Daniel Templeton
      8. YARN-4767.008.patch
        35 kB
        Daniel Templeton
      9. YARN-4767.009.patch
        36 kB
        Daniel Templeton
      10. YARN-4767.010.patch
        37 kB
        Daniel Templeton
      11. YARN-4767.011.patch
        37 kB
        Daniel Templeton

        Issue Links

          Activity

          Hide
          qwertymaniac Harsh J added a comment -

          One add-on note that we can likely also address with this one: The AmIpFilter resolves the proxy addresses to host addresses (getAllByName, getHostAddress) every single time a request is made to it, vs. caching it upfront. I think we should not try to resolve it on-request unless we have errors, cause the proxy address list does not usually change over time on an already running AM?

          Show
          qwertymaniac Harsh J added a comment - One add-on note that we can likely also address with this one: The AmIpFilter resolves the proxy addresses to host addresses (getAllByName, getHostAddress) every single time a request is made to it, vs. caching it upfront. I think we should not try to resolve it on-request unless we have errors, cause the proxy address list does not usually change over time on an already running AM?
          Hide
          templedf Daniel Templeton added a comment -

          Here's a patch that resolves the issue. The general idea is correct, but I'd love some feedback on how this patch jives with the usual web UI flow.

          Show
          templedf Daniel Templeton added a comment - Here's a patch that resolves the issue. The general idea is correct, but I'd love some feedback on how this patch jives with the usual web UI flow.
          Hide
          templedf Daniel Templeton added a comment -

          Harsh J, I think that's a worthy improvement, but I don't think it's exactly related. I found a couple of items that should be cleaned up in the current web UI code. I'll file another JIRA for those cleanup items.

          Show
          templedf Daniel Templeton added a comment - Harsh J , I think that's a worthy improvement, but I don't think it's exactly related. I found a couple of items that should be cleaned up in the current web UI code. I'll file another JIRA for those cleanup items.
          Hide
          templedf Daniel Templeton added a comment -

          Just noticed that this patch will break if the AM is on the same host as the RM. I'll post an updated patch later. For now, though, I'd still love some feedback on how I'm (ab)using the web UI framework.

          Show
          templedf Daniel Templeton added a comment - Just noticed that this patch will break if the AM is on the same host as the RM. I'll post an updated patch later. For now, though, I'd still love some feedback on how I'm (ab)using the web UI framework.
          Hide
          wilfreds Wilfred Spiegelenburg added a comment -

          In the current proxy code we have an override set that allows circular redirects.
          By default the jetty service has: http.protocol.allow-circular-redirects set to false and we override it to true to allow some failure cases to redirect. This was done in MAPREDUCE-3706. The real cause of the circular redirect was fixed in MAPREDUCE-3930.

          If we remove the change for the circular redirect again would that not fix this issue and show a error?

          Show
          wilfreds Wilfred Spiegelenburg added a comment - In the current proxy code we have an override set that allows circular redirects. By default the jetty service has: http.protocol.allow-circular-redirects set to false and we override it to true to allow some failure cases to redirect. This was done in MAPREDUCE-3706 . The real cause of the circular redirect was fixed in MAPREDUCE-3930 . If we remove the change for the circular redirect again would that not fix this issue and show a error?
          Hide
          templedf Daniel Templeton added a comment -

          Wilfred Spiegelenburg, no that won't solve the issue. The problem is that it's not a circular redirect. A circular redirect is A->A->A... This is a loop of: A contacts B and is redirected to A. From the HttpClient's perspective, there's nothing wonky going on here.

          Show
          templedf Daniel Templeton added a comment - Wilfred Spiegelenburg , no that won't solve the issue. The problem is that it's not a circular redirect. A circular redirect is A->A->A... This is a loop of: A contacts B and is redirected to A. From the HttpClient 's perspective, there's nothing wonky going on here.
          Hide
          templedf Daniel Templeton added a comment -

          This patch also works for single-node clusters. The basic idea is that the AmIpFilter appends /redirect to the URL when it redirects to the web proxy. (It only does that when it's contacted by a host other than the web proxy.) The web proxy looks for the /redirect when it receives a request. If it's there, and if the request originated from the web proxy itself, it instead redirects to an error page, ending the endless loop.

          The error page is implemented within the app page and is triggered by appending an "R" to the end of the app ID. If the app page sees the "R", it prints the error instead of the app page. This part is the part I'm least confident about being a reasonable thing to do. Feedback welcome.

          Show
          templedf Daniel Templeton added a comment - This patch also works for single-node clusters. The basic idea is that the AmIpFilter appends /redirect to the URL when it redirects to the web proxy. (It only does that when it's contacted by a host other than the web proxy.) The web proxy looks for the /redirect when it receives a request. If it's there, and if the request originated from the web proxy itself, it instead redirects to an error page, ending the endless loop. The error page is implemented within the app page and is triggered by appending an "R" to the end of the app ID. If the app page sees the "R", it prints the error instead of the app page. This part is the part I'm least confident about being a reasonable thing to do. Feedback welcome.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 9s Maven dependency ordering for branch
          +1 mvninstall 6m 39s trunk passed
          +1 compile 1m 3s trunk passed with JDK v1.8.0_74
          +1 compile 1m 16s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 24s trunk passed
          +1 mvnsite 1m 12s trunk passed
          +1 mvneclipse 0m 40s trunk passed
          +1 findbugs 2m 10s trunk passed
          +1 javadoc 0m 48s trunk passed with JDK v1.8.0_74
          +1 javadoc 0m 59s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 1m 0s the patch passed
          +1 compile 1m 1s the patch passed with JDK v1.8.0_74
          +1 javac 1m 1s the patch passed
          +1 compile 1m 13s the patch passed with JDK v1.7.0_95
          +1 javac 1m 13s the patch passed
          -1 checkstyle 0m 22s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: patch generated 3 new + 63 unchanged - 3 fixed = 66 total (was 66)
          +1 mvnsite 1m 7s the patch passed
          +1 mvneclipse 0m 32s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 40s the patch passed
          -1 javadoc 2m 11s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74 with JDK v1.8.0_74 generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100)
          +1 javadoc 0m 43s the patch passed with JDK v1.8.0_74
          +1 javadoc 0m 55s the patch passed with JDK v1.7.0_95
          +1 unit 0m 19s hadoop-yarn-server-common in the patch passed with JDK v1.8.0_74.
          -1 unit 0m 18s hadoop-yarn-server-web-proxy in the patch failed with JDK v1.8.0_74.
          -1 unit 66m 7s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
          +1 unit 0m 23s hadoop-yarn-server-common in the patch passed with JDK v1.7.0_95.
          -1 unit 0m 22s hadoop-yarn-server-web-proxy in the patch failed with JDK v1.7.0_95.
          -1 unit 68m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 20s Patch does not generate ASF License warnings.
          162m 24s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.webproxy.amfilter.TestAmFilter
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.webproxy.amfilter.TestAmFilter
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792315/YARN-4767.002.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b60e163f0251 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 2e040d3
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt
          javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10744/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10744/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 6m 39s trunk passed +1 compile 1m 3s trunk passed with JDK v1.8.0_74 +1 compile 1m 16s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 24s trunk passed +1 mvnsite 1m 12s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 2m 10s trunk passed +1 javadoc 0m 48s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 59s trunk passed with JDK v1.7.0_95 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 0s the patch passed +1 compile 1m 1s the patch passed with JDK v1.8.0_74 +1 javac 1m 1s the patch passed +1 compile 1m 13s the patch passed with JDK v1.7.0_95 +1 javac 1m 13s the patch passed -1 checkstyle 0m 22s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: patch generated 3 new + 63 unchanged - 3 fixed = 66 total (was 66) +1 mvnsite 1m 7s the patch passed +1 mvneclipse 0m 32s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 40s the patch passed -1 javadoc 2m 11s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74 with JDK v1.8.0_74 generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) +1 javadoc 0m 43s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 55s the patch passed with JDK v1.7.0_95 +1 unit 0m 19s hadoop-yarn-server-common in the patch passed with JDK v1.8.0_74. -1 unit 0m 18s hadoop-yarn-server-web-proxy in the patch failed with JDK v1.8.0_74. -1 unit 66m 7s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. +1 unit 0m 23s hadoop-yarn-server-common in the patch passed with JDK v1.7.0_95. -1 unit 0m 22s hadoop-yarn-server-web-proxy in the patch failed with JDK v1.7.0_95. -1 unit 68m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 162m 24s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.webproxy.amfilter.TestAmFilter   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.webproxy.amfilter.TestAmFilter   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792315/YARN-4767.002.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b60e163f0251 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2e040d3 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/10744/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10744/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Console output https://builds.apache.org/job/PreCommit-YARN-Build/10744/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          templedf Daniel Templeton added a comment -

          This patch should resolve the checkstyle issues. I ended up having to refactor some existing code to reduce the method sizes.

          Show
          templedf Daniel Templeton added a comment - This patch should resolve the checkstyle issues. I ended up having to refactor some existing code to reduce the method sizes.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 58s Maven dependency ordering for branch
          +1 mvninstall 6m 53s trunk passed
          +1 compile 1m 10s trunk passed with JDK v1.8.0_74
          +1 compile 1m 19s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 25s trunk passed
          +1 mvnsite 1m 16s trunk passed
          +1 mvneclipse 0m 39s trunk passed
          +1 findbugs 2m 14s trunk passed
          +1 javadoc 0m 50s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 1s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 10s Maven dependency ordering for patch
          +1 mvninstall 1m 3s the patch passed
          +1 compile 1m 9s the patch passed with JDK v1.8.0_74
          +1 javac 1m 9s the patch passed
          +1 compile 1m 16s the patch passed with JDK v1.7.0_95
          +1 javac 1m 16s the patch passed
          -1 checkstyle 0m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: patch generated 6 new + 58 unchanged - 9 fixed = 64 total (was 67)
          +1 mvnsite 1m 7s the patch passed
          +1 mvneclipse 0m 33s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 45s the patch passed
          +1 javadoc 0m 48s the patch passed with JDK v1.8.0_74
          +1 javadoc 0m 56s the patch passed with JDK v1.7.0_95
          +1 unit 0m 21s hadoop-yarn-server-common in the patch passed with JDK v1.8.0_74.
          -1 unit 0m 19s hadoop-yarn-server-web-proxy in the patch failed with JDK v1.8.0_74.
          -1 unit 72m 9s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
          +1 unit 0m 25s hadoop-yarn-server-common in the patch passed with JDK v1.7.0_95.
          -1 unit 0m 22s hadoop-yarn-server-web-proxy in the patch failed with JDK v1.7.0_95.
          -1 unit 73m 15s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          175m 42s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.webproxy.TestWebAppProxyServlet
            hadoop.yarn.server.webproxy.amfilter.TestAmFilter
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.webproxy.TestWebAppProxyServlet
            hadoop.yarn.server.webproxy.amfilter.TestAmFilter
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793979/YARN-4767.003.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 25c29b9b1702 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f84af8b
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10804/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10804/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 58s Maven dependency ordering for branch +1 mvninstall 6m 53s trunk passed +1 compile 1m 10s trunk passed with JDK v1.8.0_74 +1 compile 1m 19s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 25s trunk passed +1 mvnsite 1m 16s trunk passed +1 mvneclipse 0m 39s trunk passed +1 findbugs 2m 14s trunk passed +1 javadoc 0m 50s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 1s trunk passed with JDK v1.7.0_95 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 1m 3s the patch passed +1 compile 1m 9s the patch passed with JDK v1.8.0_74 +1 javac 1m 9s the patch passed +1 compile 1m 16s the patch passed with JDK v1.7.0_95 +1 javac 1m 16s the patch passed -1 checkstyle 0m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: patch generated 6 new + 58 unchanged - 9 fixed = 64 total (was 67) +1 mvnsite 1m 7s the patch passed +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 45s the patch passed +1 javadoc 0m 48s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 56s the patch passed with JDK v1.7.0_95 +1 unit 0m 21s hadoop-yarn-server-common in the patch passed with JDK v1.8.0_74. -1 unit 0m 19s hadoop-yarn-server-web-proxy in the patch failed with JDK v1.8.0_74. -1 unit 72m 9s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. +1 unit 0m 25s hadoop-yarn-server-common in the patch passed with JDK v1.7.0_95. -1 unit 0m 22s hadoop-yarn-server-web-proxy in the patch failed with JDK v1.7.0_95. -1 unit 73m 15s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 175m 42s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.webproxy.TestWebAppProxyServlet   hadoop.yarn.server.webproxy.amfilter.TestAmFilter   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.webproxy.TestWebAppProxyServlet   hadoop.yarn.server.webproxy.amfilter.TestAmFilter   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793979/YARN-4767.003.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 25c29b9b1702 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f84af8b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/10804/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10804/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Console output https://builds.apache.org/job/PreCommit-YARN-Build/10804/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          templedf Daniel Templeton added a comment -

          Nevermind. Looks like I have a new round of checkstyle issues to resolve, and I broke some tests.

          Show
          templedf Daniel Templeton added a comment - Nevermind. Looks like I have a new round of checkstyle issues to resolve, and I broke some tests.
          Hide
          templedf Daniel Templeton added a comment -

          This patch resolves the checkstyle issues that can be resolved and fixes the test failures.

          Show
          templedf Daniel Templeton added a comment - This patch resolves the checkstyle issues that can be resolved and fixes the test failures.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 9s Maven dependency ordering for branch
          +1 mvninstall 6m 39s trunk passed
          +1 compile 1m 1s trunk passed with JDK v1.8.0_74
          +1 compile 1m 17s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 24s trunk passed
          +1 mvnsite 1m 13s trunk passed
          +1 mvneclipse 0m 38s trunk passed
          +1 findbugs 2m 10s trunk passed
          +1 javadoc 0m 48s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 0s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 1m 2s the patch passed
          +1 compile 0m 59s the patch passed with JDK v1.8.0_74
          +1 javac 0m 59s the patch passed
          +1 compile 1m 14s the patch passed with JDK v1.7.0_95
          +1 javac 1m 14s the patch passed
          -1 checkstyle 0m 22s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: patch generated 3 new + 58 unchanged - 9 fixed = 61 total (was 67)
          +1 mvnsite 1m 6s the patch passed
          +1 mvneclipse 0m 33s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 40s the patch passed
          -1 javadoc 2m 15s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74 with JDK v1.8.0_74 generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100)
          +1 javadoc 0m 43s the patch passed with JDK v1.8.0_74
          +1 javadoc 0m 56s the patch passed with JDK v1.7.0_95
          +1 unit 0m 20s hadoop-yarn-server-common in the patch passed with JDK v1.8.0_74.
          +1 unit 0m 19s hadoop-yarn-server-web-proxy in the patch passed with JDK v1.8.0_74.
          -1 unit 68m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
          +1 unit 0m 24s hadoop-yarn-server-common in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 22s hadoop-yarn-server-web-proxy in the patch passed with JDK v1.7.0_95.
          -1 unit 70m 14s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 20s Patch does not generate ASF License warnings.
          166m 35s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794588/YARN-4767.004.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux d78b1b03e0cd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e7ed05e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt
          javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10837/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10837/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 6m 39s trunk passed +1 compile 1m 1s trunk passed with JDK v1.8.0_74 +1 compile 1m 17s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 24s trunk passed +1 mvnsite 1m 13s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 2m 10s trunk passed +1 javadoc 0m 48s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 0s trunk passed with JDK v1.7.0_95 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 2s the patch passed +1 compile 0m 59s the patch passed with JDK v1.8.0_74 +1 javac 0m 59s the patch passed +1 compile 1m 14s the patch passed with JDK v1.7.0_95 +1 javac 1m 14s the patch passed -1 checkstyle 0m 22s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: patch generated 3 new + 58 unchanged - 9 fixed = 61 total (was 67) +1 mvnsite 1m 6s the patch passed +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 40s the patch passed -1 javadoc 2m 15s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74 with JDK v1.8.0_74 generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) +1 javadoc 0m 43s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 56s the patch passed with JDK v1.7.0_95 +1 unit 0m 20s hadoop-yarn-server-common in the patch passed with JDK v1.8.0_74. +1 unit 0m 19s hadoop-yarn-server-web-proxy in the patch passed with JDK v1.8.0_74. -1 unit 68m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. +1 unit 0m 24s hadoop-yarn-server-common in the patch passed with JDK v1.7.0_95. +1 unit 0m 22s hadoop-yarn-server-web-proxy in the patch passed with JDK v1.7.0_95. -1 unit 70m 14s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 166m 35s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794588/YARN-4767.004.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d78b1b03e0cd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e7ed05e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10837/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Console output https://builds.apache.org/job/PreCommit-YARN-Build/10837/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ashwinshankar77 Ashwin Shankar added a comment -

          At Netflix we are hitting this issue pretty consistently and users can't get to their jobs. Committers, could you please take a look at the patch?

          Show
          ashwinshankar77 Ashwin Shankar added a comment - At Netflix we are hitting this issue pretty consistently and users can't get to their jobs. Committers, could you please take a look at the patch?
          Hide
          templedf Daniel Templeton added a comment -

          I rebased the patch, just in case.

          Show
          templedf Daniel Templeton added a comment - I rebased the patch, just in case.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 10s Maven dependency ordering for branch
          +1 mvninstall 5m 55s trunk passed
          +1 compile 1m 8s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 1m 7s trunk passed
          +1 mvneclipse 0m 34s trunk passed
          +1 findbugs 1m 41s trunk passed
          +1 javadoc 0m 50s trunk passed
          0 mvndep 0m 8s Maven dependency ordering for patch
          +1 mvninstall 0m 54s the patch passed
          +1 compile 1m 7s the patch passed
          +1 javac 1m 7s the patch passed
          -1 checkstyle 0m 25s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: patch generated 3 new + 64 unchanged - 10 fixed = 67 total (was 74)
          +1 mvnsite 1m 0s the patch passed
          +1 mvneclipse 0m 28s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 58s the patch passed
          -1 javadoc 1m 55s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 2 new + 1273 unchanged - 0 fixed = 1275 total (was 1273)
          +1 javadoc 0m 44s the patch passed
          +1 unit 0m 21s hadoop-yarn-server-common in the patch passed.
          +1 unit 0m 18s hadoop-yarn-server-web-proxy in the patch passed.
          -1 unit 29m 17s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 16s Patch does not generate ASF License warnings.
          49m 45s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestRMAdminService



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806038/YARN-4767.005.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 9dca8e04fac9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / edd716e
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11681/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt
          javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common: https://builds.apache.org/job/PreCommit-YARN-Build/11681/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11681/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11681/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11681/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11681/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 5m 55s trunk passed +1 compile 1m 8s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 7s trunk passed +1 mvneclipse 0m 34s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 50s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 0m 54s the patch passed +1 compile 1m 7s the patch passed +1 javac 1m 7s the patch passed -1 checkstyle 0m 25s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: patch generated 3 new + 64 unchanged - 10 fixed = 67 total (was 74) +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 28s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 58s the patch passed -1 javadoc 1m 55s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 2 new + 1273 unchanged - 0 fixed = 1275 total (was 1273) +1 javadoc 0m 44s the patch passed +1 unit 0m 21s hadoop-yarn-server-common in the patch passed. +1 unit 0m 18s hadoop-yarn-server-web-proxy in the patch passed. -1 unit 29m 17s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 16s Patch does not generate ASF License warnings. 49m 45s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestRMAdminService Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806038/YARN-4767.005.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9dca8e04fac9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / edd716e Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11681/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common: https://builds.apache.org/job/PreCommit-YARN-Build/11681/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11681/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11681/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11681/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Console output https://builds.apache.org/job/PreCommit-YARN-Build/11681/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          templedf Daniel Templeton added a comment -

          Here's one more patch that kills one more of the checkstyle issues. The last two are line length complaints on lines that cannot be shortened without compromising readability.

          Ashwin Shankar, I'd love to hear more about what you're encountering. Have you tried applying this patch in your environment? The scenario where the redirect loop happens is pretty unusual. How come you're seeing it so often?

          Show
          templedf Daniel Templeton added a comment - Here's one more patch that kills one more of the checkstyle issues. The last two are line length complaints on lines that cannot be shortened without compromising readability. Ashwin Shankar , I'd love to hear more about what you're encountering. Have you tried applying this patch in your environment? The scenario where the redirect loop happens is pretty unusual. How come you're seeing it so often?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 13s Maven dependency ordering for branch
          +1 mvninstall 7m 18s trunk passed
          +1 compile 1m 33s trunk passed
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 1m 17s trunk passed
          +1 mvneclipse 0m 37s trunk passed
          +1 findbugs 2m 4s trunk passed
          +1 javadoc 1m 1s trunk passed
          0 mvndep 0m 8s Maven dependency ordering for patch
          +1 mvninstall 1m 8s the patch passed
          +1 compile 1m 42s the patch passed
          +1 javac 1m 42s the patch passed
          -1 checkstyle 0m 29s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: patch generated 2 new + 65 unchanged - 10 fixed = 67 total (was 75)
          +1 mvnsite 1m 12s the patch passed
          +1 mvneclipse 0m 32s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 50s the patch passed
          -1 javadoc 2m 21s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 2 new + 1273 unchanged - 0 fixed = 1275 total (was 1273)
          +1 javadoc 0m 47s the patch passed
          +1 unit 0m 22s hadoop-yarn-server-common in the patch passed.
          +1 unit 0m 18s hadoop-yarn-server-web-proxy in the patch passed.
          -1 unit 30m 14s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          55m 51s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806128/YARN-4767.006.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 0d090cc6a122 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9a31e5d
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11683/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt
          javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common: https://builds.apache.org/job/PreCommit-YARN-Build/11683/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11683/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11683/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11683/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11683/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 13s Maven dependency ordering for branch +1 mvninstall 7m 18s trunk passed +1 compile 1m 33s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 1m 17s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 2m 4s trunk passed +1 javadoc 1m 1s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 1m 8s the patch passed +1 compile 1m 42s the patch passed +1 javac 1m 42s the patch passed -1 checkstyle 0m 29s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: patch generated 2 new + 65 unchanged - 10 fixed = 67 total (was 75) +1 mvnsite 1m 12s the patch passed +1 mvneclipse 0m 32s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 50s the patch passed -1 javadoc 2m 21s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 2 new + 1273 unchanged - 0 fixed = 1275 total (was 1273) +1 javadoc 0m 47s the patch passed +1 unit 0m 22s hadoop-yarn-server-common in the patch passed. +1 unit 0m 18s hadoop-yarn-server-web-proxy in the patch passed. -1 unit 30m 14s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 55m 51s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806128/YARN-4767.006.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0d090cc6a122 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9a31e5d Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11683/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common: https://builds.apache.org/job/PreCommit-YARN-Build/11683/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11683/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11683/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11683/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Console output https://builds.apache.org/job/PreCommit-YARN-Build/11683/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment - - edited

          The patch looks good to me, but for the following nits:

          1. More an observation. We seem to be appending "R" at the end to capture a redirection. It seems a little hacky. The alternative of defining a special class with a boolean to capture redirection seems rather heavy and unnecessary. Can we add a more simple test that verifies this addition of "R" so it is easy for the developer changing surrounding code in the future.
          2. AppBlock#generateOverviewTable: The ternary operators aren't following the coding conventions and are also hard to read. Mind updating them to make sure "?" and the associated value are on the same line, and ":" and the associated value are on the same line? And, for nested ternary operator, additional indentation is good. For instance,
                    app.getTrackingUrl() == null || app.getTrackingUrl().equals(UNAVAILABLE) 
                        ? null 
                        : root_url(app.getTrackingUrl()),
                    app.getTrackingUrl() == null || app.getTrackingUrl().equals(UNAVAILABLE)
                        ? "Unassigned"
                        : app.getAppState() == YarnApplicationState.FINISHED ||
                          app.getAppState() == YarnApplicationState.FAILED ||
                          app.getAppState() == YarnApplicationState.KILLED
                          ? "History"
                          : "ApplicationMaster");
            

          Xuan Gong, Vinod Kumar Vavilapalli - you are more familiar with the redirection and proxy code. Mind taking a quick look? I think we should try and get this into 2.8.0.

          Show
          kasha Karthik Kambatla added a comment - - edited The patch looks good to me, but for the following nits: More an observation. We seem to be appending "R" at the end to capture a redirection. It seems a little hacky. The alternative of defining a special class with a boolean to capture redirection seems rather heavy and unnecessary. Can we add a more simple test that verifies this addition of "R" so it is easy for the developer changing surrounding code in the future. AppBlock#generateOverviewTable: The ternary operators aren't following the coding conventions and are also hard to read. Mind updating them to make sure "?" and the associated value are on the same line, and ":" and the associated value are on the same line? And, for nested ternary operator, additional indentation is good. For instance, app.getTrackingUrl() == null || app.getTrackingUrl().equals(UNAVAILABLE) ? null : root_url(app.getTrackingUrl()), app.getTrackingUrl() == null || app.getTrackingUrl().equals(UNAVAILABLE) ? "Unassigned" : app.getAppState() == YarnApplicationState.FINISHED || app.getAppState() == YarnApplicationState.FAILED || app.getAppState() == YarnApplicationState.KILLED ? "History" : "ApplicationMaster" ); Xuan Gong , Vinod Kumar Vavilapalli - you are more familiar with the redirection and proxy code. Mind taking a quick look? I think we should try and get this into 2.8.0.
          Hide
          vigith Vigith Maurice added a comment -

          Can't we just put a TTL in the header and decrement it for every redirect, once it hits zero we know that we have redirected enough ?

          Show
          vigith Vigith Maurice added a comment - Can't we just put a TTL in the header and decrement it for every redirect, once it hits zero we know that we have redirected enough ?
          Hide
          templedf Daniel Templeton added a comment -

          Not that simple. It's not a single connection that's looping. Each
          connection is only redirected once. The pattern is: new connection ->
          redirect -> new connection -> redirect...

          Show
          templedf Daniel Templeton added a comment - Not that simple. It's not a single connection that's looping. Each connection is only redirected once. The pattern is: new connection -> redirect -> new connection -> redirect...
          Hide
          vigith Vigith Maurice added a comment -

          when we make the new connection, can't we put in the new connection header the TTL from the redirect after decrementing it?

          Show
          vigith Vigith Maurice added a comment - when we make the new connection, can't we put in the new connection header the TTL from the redirect after decrementing it?
          Hide
          templedf Daniel Templeton added a comment -

          The full trace looks like this:

          1. Thread 1) Proxy receives incoming request
          2. Thread 1) Proxy opens connection to the NM
          3. Thread 1) NM sends redirect back to the proxy
          4. Thread 1) Proxy opens a new connection to the proxy
          5. Thread 2) Proxy receives incoming request
          6. Thread 2) Proxy opens connection to the NM
          7. Thread 2) NM sends redirect back to the proxy
          8. Thread 2) Proxy opens a new connection to the proxy

          And so on.

          I'm not aware of a TTL at the HTTP level. It's technically possible to use an HTTP header to flag that the incoming connection is a redirect rather than encoding it in the URL, but it would be a pretty invasive change. The code that handles the redirects is buried pretty deep. I think it may even be in a bundled library. Getting to it to make it respect the header probably isn't worth it.

          Show
          templedf Daniel Templeton added a comment - The full trace looks like this: Thread 1) Proxy receives incoming request Thread 1) Proxy opens connection to the NM Thread 1) NM sends redirect back to the proxy Thread 1) Proxy opens a new connection to the proxy Thread 2) Proxy receives incoming request Thread 2) Proxy opens connection to the NM Thread 2) NM sends redirect back to the proxy Thread 2) Proxy opens a new connection to the proxy And so on. I'm not aware of a TTL at the HTTP level. It's technically possible to use an HTTP header to flag that the incoming connection is a redirect rather than encoding it in the URL, but it would be a pretty invasive change. The code that handles the redirects is buried pretty deep. I think it may even be in a bundled library. Getting to it to make it respect the header probably isn't worth it.
          Hide
          templedf Daniel Templeton added a comment -

          Here's a patch that cleans up those silly nested ternary operators. As soon as we get some consensus that this approach is the right approach, I'll work on adding tests around the redirect tagging. I agree with Karthik Kambatla that the current approach is hacky, but I don't see a less hacky approach at the moment that is not overkill.

          Show
          templedf Daniel Templeton added a comment - Here's a patch that cleans up those silly nested ternary operators. As soon as we get some consensus that this approach is the right approach, I'll work on adding tests around the redirect tagging. I agree with Karthik Kambatla that the current approach is hacky, but I don't see a less hacky approach at the moment that is not overkill.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 7s Maven dependency ordering for branch
          +1 mvninstall 6m 11s trunk passed
          +1 compile 1m 10s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 1m 6s trunk passed
          +1 mvneclipse 0m 34s trunk passed
          +1 findbugs 1m 45s trunk passed
          +1 javadoc 0m 46s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 0m 56s the patch passed
          +1 compile 1m 11s the patch passed
          +1 javac 1m 11s the patch passed
          -1 checkstyle 0m 24s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 5 new + 61 unchanged - 13 fixed = 66 total (was 74)
          +1 mvnsite 1m 2s the patch passed
          +1 mvneclipse 0m 28s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 6s the patch passed
          -1 javadoc 0m 12s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 2 new + 163 unchanged - 0 fixed = 165 total (was 163)
          +1 unit 0m 22s hadoop-yarn-server-common in the patch passed.
          +1 unit 0m 19s hadoop-yarn-server-web-proxy in the patch passed.
          -1 unit 35m 32s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          56m 41s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807544/YARN-4767.007.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7033d073b567 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 16b1cc7
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11801/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11801/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11801/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11801/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11801/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11801/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 7s Maven dependency ordering for branch +1 mvninstall 6m 11s trunk passed +1 compile 1m 10s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 34s trunk passed +1 findbugs 1m 45s trunk passed +1 javadoc 0m 46s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 0m 56s the patch passed +1 compile 1m 11s the patch passed +1 javac 1m 11s the patch passed -1 checkstyle 0m 24s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 5 new + 61 unchanged - 13 fixed = 66 total (was 74) +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 28s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 6s the patch passed -1 javadoc 0m 12s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 2 new + 163 unchanged - 0 fixed = 165 total (was 163) +1 unit 0m 22s hadoop-yarn-server-common in the patch passed. +1 unit 0m 19s hadoop-yarn-server-web-proxy in the patch passed. -1 unit 35m 32s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 56m 41s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807544/YARN-4767.007.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7033d073b567 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 16b1cc7 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11801/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11801/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11801/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11801/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11801/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Console output https://builds.apache.org/job/PreCommit-YARN-Build/11801/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ashwinshankar77 Ashwin Shankar added a comment - - edited

          hey Daniel Templeton,
          Thanks much for rebasing the patch! Just to give you some context on what we see at my company - we first got complaints from users that they cannot access the AM UI.Since these http requests go through the Web proxy, we looked at that process and found that it was unresponsive since all its threads were busy.When we listed open file descriptors, we saw that the webproxy had many connections from itself to itself, which seemed weird then, but makes sense now since its due to AM redirecting requests back to proxy. Web proxy logs showed that most of the requests were made to one or two specific apps. We then looked at that app's AM logs and found "UnknownHostException" when AM was trying to resolve proxy host(which is basically the master node where RM runs) in AmIpFilter code. We believe it wasn't able to resolve due to an intermittent network event to the DNS but its not conclusive. Overall, this issue has been occurring pretty much once every week and we have to bounce the webproxy to fix it.

          Thanks Karthik Kambatla for the review! Xuan Gong, Vinod Kumar Vavilapalli please take a look at the patch when you get a chance. we would like to backport it as soon as its committed.

          Show
          ashwinshankar77 Ashwin Shankar added a comment - - edited hey Daniel Templeton , Thanks much for rebasing the patch! Just to give you some context on what we see at my company - we first got complaints from users that they cannot access the AM UI.Since these http requests go through the Web proxy, we looked at that process and found that it was unresponsive since all its threads were busy.When we listed open file descriptors, we saw that the webproxy had many connections from itself to itself, which seemed weird then, but makes sense now since its due to AM redirecting requests back to proxy. Web proxy logs showed that most of the requests were made to one or two specific apps. We then looked at that app's AM logs and found "UnknownHostException" when AM was trying to resolve proxy host(which is basically the master node where RM runs) in AmIpFilter code. We believe it wasn't able to resolve due to an intermittent network event to the DNS but its not conclusive. Overall, this issue has been occurring pretty much once every week and we have to bounce the webproxy to fix it. Thanks Karthik Kambatla for the review! Xuan Gong , Vinod Kumar Vavilapalli please take a look at the patch when you get a chance. we would like to backport it as soon as its committed.
          Hide
          templedf Daniel Templeton added a comment -

          Ping Xuan Gong, Vinod Kumar Vavilapalli. Would love feedback on the approach in this patch. Thanks!

          Show
          templedf Daniel Templeton added a comment - Ping Xuan Gong , Vinod Kumar Vavilapalli . Would love feedback on the approach in this patch. Thanks!
          Hide
          templedf Daniel Templeton added a comment -

          Flood ping, Xuan Gong, Vinod Kumar Vavilapalli. It would be great to get some feedback before we talk about committing.

          Show
          templedf Daniel Templeton added a comment - Flood ping, Xuan Gong , Vinod Kumar Vavilapalli . It would be great to get some feedback before we talk about committing.
          Hide
          kasha Karthik Kambatla added a comment -

          Daniel Templeton - I am comfortable with the approach here. I have some nits to point out, but will post the comments along with review of tests. Can we add tests and make progress towards getting this in?

          Xuan Gong and Vinod Kumar Vavilapalli - chime in if you can. I will interpret your silence as a go-ahead

          Show
          kasha Karthik Kambatla added a comment - Daniel Templeton - I am comfortable with the approach here. I have some nits to point out, but will post the comments along with review of tests. Can we add tests and make progress towards getting this in? Xuan Gong and Vinod Kumar Vavilapalli - chime in if you can. I will interpret your silence as a go-ahead
          Hide
          yangyishan0901m Yishan Yang added a comment -

          Is that possible to provide a patch for hadoop 2.7.2, it's kind of hard to back port this patch to 2.7.2 version. Since 2.8.0 won't be released shortly. Thanks

          Show
          yangyishan0901m Yishan Yang added a comment - Is that possible to provide a patch for hadoop 2.7.2, it's kind of hard to back port this patch to 2.7.2 version. Since 2.8.0 won't be released shortly. Thanks
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Apologies Daniel Templeton, I haven't been looking at this.

          I also ran into the same issue a couple of months back, but didn't see this JIRA.

          Just looked at the patch. It took me a little while, but I understood the problem and your patch.

          The approach overall looks good to me.

          I am sure you had to do a bit of testing and bringing that back may be some effort, but I got few comments

          • The "append-/redirect" approach will fail if the AM URIs have query parameters.
            • This is because, on the WebAppProxyServlet side -> methodAction(), it looks at HttpServletRequest.getPathInfo() which "returns any extra path information associated with the URL the client sent when it made this request. The extra path information follows the servlet path but precedes the query string and will start with a / character.".
            • How about we instead go with a String target = redirectUrl + "/redirect" + httpReq.getRequestURI(); approach? Or something like that as a prefix..
          • Also if we do that, we won't need (a) redirecting to the app-page (b) the internal "R" hack to let the app-pages know (c) changing AppBlock etc? We can simply look for paths of the form "http://RM-proxyAddress/proxy/cluster/application_id/redirect" and treat them separately by sending them to an error-page?
          • Also inject the 'wrong' proxy-address and add a test-case which proves the patch?

          Let me know how I can help in push this forward.

          Also looking at a 2.8.0 RC0 this weekend / beginning next week - let's see if we can get this moving before that?

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Apologies Daniel Templeton , I haven't been looking at this. I also ran into the same issue a couple of months back, but didn't see this JIRA. Just looked at the patch. It took me a little while, but I understood the problem and your patch. The approach overall looks good to me. I am sure you had to do a bit of testing and bringing that back may be some effort, but I got few comments The "append-/redirect" approach will fail if the AM URIs have query parameters. This is because, on the WebAppProxyServlet side -> methodAction() , it looks at HttpServletRequest.getPathInfo() which "returns any extra path information associated with the URL the client sent when it made this request. The extra path information follows the servlet path but precedes the query string and will start with a / character." . How about we instead go with a String target = redirectUrl + "/redirect" + httpReq.getRequestURI(); approach? Or something like that as a prefix.. Also if we do that, we won't need (a) redirecting to the app-page (b) the internal "R" hack to let the app-pages know (c) changing AppBlock etc? We can simply look for paths of the form "http://RM-proxyAddress/proxy/cluster/application_id/redirect" and treat them separately by sending them to an error-page? Also inject the 'wrong' proxy-address and add a test-case which proves the patch? Let me know how I can help in push this forward. Also looking at a 2.8.0 RC0 this weekend / beginning next week - let's see if we can get this moving before that?
          Hide
          templedf Daniel Templeton added a comment -

          Thanks, Vinod Kumar Vavilapalli. Here's a patch that should address your comments.

          Show
          templedf Daniel Templeton added a comment - Thanks, Vinod Kumar Vavilapalli . Here's a patch that should address your comments.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 9s Maven dependency ordering for branch
          +1 mvninstall 7m 6s trunk passed
          +1 compile 2m 26s trunk passed
          +1 checkstyle 0m 39s trunk passed
          +1 mvnsite 1m 50s trunk passed
          +1 mvneclipse 0m 57s trunk passed
          +1 findbugs 2m 52s trunk passed
          +1 javadoc 1m 18s trunk passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 1m 29s the patch passed
          +1 compile 2m 25s the patch passed
          +1 javac 2m 25s the patch passed
          -1 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn: The patch generated 10 new + 122 unchanged - 10 fixed = 132 total (was 132)
          +1 mvnsite 1m 48s the patch passed
          +1 mvneclipse 0m 48s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 20s the patch passed
          -1 javadoc 0m 20s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 3 new + 961 unchanged - 0 fixed = 964 total (was 961)
          +1 unit 2m 21s hadoop-yarn-common in the patch passed.
          +1 unit 0m 29s hadoop-yarn-server-common in the patch passed.
          -1 unit 0m 22s hadoop-yarn-server-web-proxy in the patch failed.
          +1 unit 38m 15s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          72m 12s



          Reason Tests
          Failed junit tests hadoop.yarn.server.webproxy.TestWebAppProxyServlet
            hadoop.yarn.server.webproxy.amfilter.TestAmFilter



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825360/YARN-4767.008.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c137c8b124b6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a1f3293
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12883/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12883/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12883/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12883/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12883/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12883/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 7m 6s trunk passed +1 compile 2m 26s trunk passed +1 checkstyle 0m 39s trunk passed +1 mvnsite 1m 50s trunk passed +1 mvneclipse 0m 57s trunk passed +1 findbugs 2m 52s trunk passed +1 javadoc 1m 18s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 29s the patch passed +1 compile 2m 25s the patch passed +1 javac 2m 25s the patch passed -1 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn: The patch generated 10 new + 122 unchanged - 10 fixed = 132 total (was 132) +1 mvnsite 1m 48s the patch passed +1 mvneclipse 0m 48s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 20s the patch passed -1 javadoc 0m 20s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 3 new + 961 unchanged - 0 fixed = 964 total (was 961) +1 unit 2m 21s hadoop-yarn-common in the patch passed. +1 unit 0m 29s hadoop-yarn-server-common in the patch passed. -1 unit 0m 22s hadoop-yarn-server-web-proxy in the patch failed. +1 unit 38m 15s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 72m 12s Reason Tests Failed junit tests hadoop.yarn.server.webproxy.TestWebAppProxyServlet   hadoop.yarn.server.webproxy.amfilter.TestAmFilter Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825360/YARN-4767.008.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c137c8b124b6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a1f3293 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12883/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12883/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12883/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12883/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12883/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12883/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Daniel Templeton, the latest patch looks much better to me.

          Also like that the unit tests verify the added code in a pin-pointy way.

          There's one bug in the code

              this.failurePageUrlBase =
                  StringHelper.pjoin(WebAppUtils.getResolvedRMWebAppURLWithScheme(conf),
                    "cluster", "failure");
          

          You put cluster prefix here, but don't expect that prefix in other places.

          Can you please address Jenkins warnings too?

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Daniel Templeton , the latest patch looks much better to me. Also like that the unit tests verify the added code in a pin-pointy way. There's one bug in the code this .failurePageUrlBase = StringHelper.pjoin(WebAppUtils.getResolvedRMWebAppURLWithScheme(conf), "cluster" , "failure" ); You put cluster prefix here, but don't expect that prefix in other places. Can you please address Jenkins warnings too?
          Hide
          templedf Daniel Templeton added a comment - - edited

          This patch addresses the checkstyle issues. The javadoc complaints are sadly unavoidable.

          The "cluster" in the failure page path is correct. See the rmAppPageUrlBase above it. Without the "cluster" there, the patch doesn't work. Do you mean that I should be testing with the "cluster" prefix in the test files? Since the redirection bit doesn't actually care what the path is, I wasn't, but if it will brighten your day, I can do that.

          Show
          templedf Daniel Templeton added a comment - - edited This patch addresses the checkstyle issues. The javadoc complaints are sadly unavoidable. The "cluster" in the failure page path is correct. See the rmAppPageUrlBase above it. Without the "cluster" there, the patch doesn't work. Do you mean that I should be testing with the "cluster" prefix in the test files? Since the redirection bit doesn't actually care what the path is, I wasn't, but if it will brighten your day, I can do that.
          Hide
          templedf Daniel Templeton added a comment -

          Actually, I am testing with the "cluster" prefix in TestWebAppProxyServet. In these lines:

                String expected =
                    WebAppUtils.getResolvedRMWebAppURLWithScheme(configuration)
                      + "/failure/application_00_0";
          

          the WebAppUtils.getResolvedRMWebAppURLWithScheme(configuration) call adds the prefix, so the expected path is "/cluster/failure/application_00_0".

          Show
          templedf Daniel Templeton added a comment - Actually, I am testing with the "cluster" prefix in TestWebAppProxyServet . In these lines: String expected = WebAppUtils.getResolvedRMWebAppURLWithScheme(configuration) + "/failure/application_00_0" ; the WebAppUtils.getResolvedRMWebAppURLWithScheme(configuration) call adds the prefix, so the expected path is "/cluster/failure/application_00_0".
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Yes, then the tests should be fixed - they are failing now.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Yes, then the tests should be fixed - they are failing now.
          Hide
          templedf Daniel Templeton added a comment - - edited

          Oh, whoops. Missed the unit test failure. Huh. It passes for me locally. I'll take a look. (Correction: I was passing for me locally. I must have broken it without noticing. Looks like you're right...)

          Show
          templedf Daniel Templeton added a comment - - edited Oh, whoops. Missed the unit test failure. Huh. It passes for me locally. I'll take a look. (Correction: I was passing for me locally. I must have broken it without noticing. Looks like you're right...)
          Hide
          templedf Daniel Templeton added a comment -

          Here's a patch that fixes the test failure.

          Show
          templedf Daniel Templeton added a comment - Here's a patch that fixes the test failure.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 11s Maven dependency ordering for branch
          +1 mvninstall 7m 52s trunk passed
          +1 compile 2m 21s trunk passed
          +1 checkstyle 0m 38s trunk passed
          +1 mvnsite 1m 25s trunk passed
          +1 mvneclipse 0m 42s trunk passed
          +1 findbugs 2m 14s trunk passed
          +1 javadoc 0m 59s trunk passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 1m 9s the patch passed
          +1 compile 2m 14s the patch passed
          +1 javac 2m 14s the patch passed
          +1 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 91 unchanged - 4 fixed = 91 total (was 95)
          +1 mvnsite 1m 33s the patch passed
          +1 mvneclipse 0m 40s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 5s the patch passed
          -1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 3 new + 961 unchanged - 0 fixed = 964 total (was 961)
          +1 unit 2m 25s hadoop-yarn-common in the patch passed.
          -1 unit 0m 25s hadoop-yarn-server-web-proxy in the patch failed.
          +1 unit 34m 21s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          65m 39s



          Reason Tests
          Failed junit tests hadoop.yarn.server.webproxy.TestWebAppProxyServlet
            hadoop.yarn.server.webproxy.amfilter.TestAmFilter



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826071/YARN-4767.009.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2cc331d5d738 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 5d1609d
          Default Java 1.8.0_101
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12928/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12928/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12928/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12928/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12928/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 7m 52s trunk passed +1 compile 2m 21s trunk passed +1 checkstyle 0m 38s trunk passed +1 mvnsite 1m 25s trunk passed +1 mvneclipse 0m 42s trunk passed +1 findbugs 2m 14s trunk passed +1 javadoc 0m 59s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 9s the patch passed +1 compile 2m 14s the patch passed +1 javac 2m 14s the patch passed +1 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 91 unchanged - 4 fixed = 91 total (was 95) +1 mvnsite 1m 33s the patch passed +1 mvneclipse 0m 40s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 5s the patch passed -1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 3 new + 961 unchanged - 0 fixed = 964 total (was 961) +1 unit 2m 25s hadoop-yarn-common in the patch passed. -1 unit 0m 25s hadoop-yarn-server-web-proxy in the patch failed. +1 unit 34m 21s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 65m 39s Reason Tests Failed junit tests hadoop.yarn.server.webproxy.TestWebAppProxyServlet   hadoop.yarn.server.webproxy.amfilter.TestAmFilter Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826071/YARN-4767.009.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2cc331d5d738 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5d1609d Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12928/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12928/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12928/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12928/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12928/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 10s Maven dependency ordering for branch
          +1 mvninstall 8m 14s trunk passed
          +1 compile 2m 52s trunk passed
          +1 checkstyle 0m 44s trunk passed
          +1 mvnsite 1m 40s trunk passed
          +1 mvneclipse 0m 46s trunk passed
          +1 findbugs 2m 35s trunk passed
          +1 javadoc 1m 1s trunk passed
          0 mvndep 0m 10s Maven dependency ordering for patch
          +1 mvninstall 1m 22s the patch passed
          +1 compile 2m 47s the patch passed
          +1 javac 2m 47s the patch passed
          +1 checkstyle 0m 43s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 92 unchanged - 4 fixed = 92 total (was 96)
          +1 mvnsite 1m 33s the patch passed
          +1 mvneclipse 0m 39s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 55s the patch passed
          -1 javadoc 0m 20s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 3 new + 961 unchanged - 0 fixed = 964 total (was 961)
          +1 unit 2m 25s hadoop-yarn-common in the patch passed.
          -1 unit 0m 23s hadoop-yarn-server-web-proxy in the patch failed.
          -1 unit 41m 22s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          74m 58s



          Reason Tests
          Failed junit tests hadoop.yarn.server.webproxy.amfilter.TestAmFilter
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826075/YARN-4767.010.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux d1ded9d54278 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 5d1609d
          Default Java 1.8.0_101
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12929/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12929/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12929/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12929/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt https://builds.apache.org/job/PreCommit-YARN-Build/12929/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12929/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12929/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 8m 14s trunk passed +1 compile 2m 52s trunk passed +1 checkstyle 0m 44s trunk passed +1 mvnsite 1m 40s trunk passed +1 mvneclipse 0m 46s trunk passed +1 findbugs 2m 35s trunk passed +1 javadoc 1m 1s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 1m 22s the patch passed +1 compile 2m 47s the patch passed +1 javac 2m 47s the patch passed +1 checkstyle 0m 43s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 92 unchanged - 4 fixed = 92 total (was 96) +1 mvnsite 1m 33s the patch passed +1 mvneclipse 0m 39s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 55s the patch passed -1 javadoc 0m 20s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 3 new + 961 unchanged - 0 fixed = 964 total (was 961) +1 unit 2m 25s hadoop-yarn-common in the patch passed. -1 unit 0m 23s hadoop-yarn-server-web-proxy in the patch failed. -1 unit 41m 22s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 74m 58s Reason Tests Failed junit tests hadoop.yarn.server.webproxy.amfilter.TestAmFilter   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826075/YARN-4767.010.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d1ded9d54278 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5d1609d Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12929/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12929/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12929/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12929/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt https://builds.apache.org/job/PreCommit-YARN-Build/12929/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12929/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12929/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 12s Maven dependency ordering for branch
          +1 mvninstall 8m 0s trunk passed
          +1 compile 3m 1s trunk passed
          +1 checkstyle 0m 45s trunk passed
          +1 mvnsite 1m 44s trunk passed
          +1 mvneclipse 0m 47s trunk passed
          +1 findbugs 2m 31s trunk passed
          +1 javadoc 1m 14s trunk passed
          0 mvndep 0m 10s Maven dependency ordering for patch
          +1 mvninstall 1m 26s the patch passed
          +1 compile 2m 50s the patch passed
          +1 javac 2m 50s the patch passed
          +1 checkstyle 0m 44s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 92 unchanged - 4 fixed = 92 total (was 96)
          +1 mvnsite 1m 36s the patch passed
          +1 mvneclipse 0m 42s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 4s the patch passed
          -1 javadoc 0m 27s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 3 new + 961 unchanged - 0 fixed = 964 total (was 961)
          +1 unit 2m 40s hadoop-yarn-common in the patch passed.
          +1 unit 0m 25s hadoop-yarn-server-web-proxy in the patch passed.
          +1 unit 38m 43s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          73m 32s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826091/YARN-4767.010.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 345008d169eb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8b57be1
          Default Java 1.8.0_101
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12932/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12932/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12932/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 8m 0s trunk passed +1 compile 3m 1s trunk passed +1 checkstyle 0m 45s trunk passed +1 mvnsite 1m 44s trunk passed +1 mvneclipse 0m 47s trunk passed +1 findbugs 2m 31s trunk passed +1 javadoc 1m 14s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 1m 26s the patch passed +1 compile 2m 50s the patch passed +1 javac 2m 50s the patch passed +1 checkstyle 0m 44s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 92 unchanged - 4 fixed = 92 total (was 96) +1 mvnsite 1m 36s the patch passed +1 mvneclipse 0m 42s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 4s the patch passed -1 javadoc 0m 27s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 3 new + 961 unchanged - 0 fixed = 964 total (was 961) +1 unit 2m 40s hadoop-yarn-common in the patch passed. +1 unit 0m 25s hadoop-yarn-server-web-proxy in the patch passed. +1 unit 38m 43s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 73m 32s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826091/YARN-4767.010.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 345008d169eb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8b57be1 Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12932/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12932/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12932/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          Vinod Kumar Vavilapalli - what do you think of the latest patch? I can take care of the commit once you feel comfortable with the patch.

          Show
          kasha Karthik Kambatla added a comment - Vinod Kumar Vavilapalli - what do you think of the latest patch? I can take care of the commit once you feel comfortable with the patch.
          Hide
          kasha Karthik Kambatla added a comment -
          Show
          kasha Karthik Kambatla added a comment - Ping Vinod Kumar Vavilapalli
          Hide
          kasha Karthik Kambatla added a comment -

          The latest patch looks good to me. Daniel Templeton - is the javadoc related, and does it need fixing?

          Vinod Kumar Vavilapalli - from your comments here, I take it you are comfortable with the approach and wanted the tests to be fixed. Would be nice if you could take another look, but otherwise let me assume you are a +0 and commit it once the javadocs are cleared.

          Show
          kasha Karthik Kambatla added a comment - The latest patch looks good to me. Daniel Templeton - is the javadoc related, and does it need fixing? Vinod Kumar Vavilapalli - from your comments here, I take it you are comfortable with the approach and wanted the tests to be fixed. Would be nice if you could take another look, but otherwise let me assume you are a +0 and commit it once the javadocs are cleared.
          Hide
          templedf Daniel Templeton added a comment -

          The javadoc complaints are all about the fact that _ is used as a variable name. I'd really love to fix that, but it's way out of scope for this patch.

          Show
          templedf Daniel Templeton added a comment - The javadoc complaints are all about the fact that _ is used as a variable name. I'd really love to fix that, but it's way out of scope for this patch.
          Hide
          kasha Karthik Kambatla added a comment -

          +1. Will commit this later today. /cc Vinod Kumar Vavilapalli

          Show
          kasha Karthik Kambatla added a comment - +1. Will commit this later today. /cc Vinod Kumar Vavilapalli
          Hide
          kasha Karthik Kambatla added a comment -

          Apologies for not committing it last week before I went away. Now, the patch does not apply cleanly. Daniel Templeton - could you please upload another patch. Will try my best to commit it soon this time around.

          Show
          kasha Karthik Kambatla added a comment - Apologies for not committing it last week before I went away. Now, the patch does not apply cleanly. Daniel Templeton - could you please upload another patch. Will try my best to commit it soon this time around.
          Hide
          templedf Daniel Templeton added a comment -

          Rebased.

          Show
          templedf Daniel Templeton added a comment - Rebased.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 11s Maven dependency ordering for branch
          +1 mvninstall 8m 8s trunk passed
          +1 compile 2m 50s trunk passed
          +1 checkstyle 0m 50s trunk passed
          +1 mvnsite 1m 36s trunk passed
          +1 mvneclipse 0m 44s trunk passed
          +1 findbugs 2m 31s trunk passed
          +1 javadoc 1m 1s trunk passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 1m 24s the patch passed
          +1 compile 2m 49s the patch passed
          +1 javac 2m 49s the patch passed
          +1 checkstyle 0m 43s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 91 unchanged - 4 fixed = 91 total (was 95)
          +1 mvnsite 1m 34s the patch passed
          +1 mvneclipse 0m 42s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 2s the patch passed
          -1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 3 new + 935 unchanged - 0 fixed = 938 total (was 935)
          +1 unit 2m 36s hadoop-yarn-common in the patch passed.
          +1 unit 0m 24s hadoop-yarn-server-web-proxy in the patch passed.
          -1 unit 37m 5s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          70m 50s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831349/YARN-4767.011.patch
          JIRA Issue YARN-4767
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c6bbbdff516e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 607705c
          Default Java 1.8.0_101
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13267/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/13267/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13267/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13267/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13267/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 8m 8s trunk passed +1 compile 2m 50s trunk passed +1 checkstyle 0m 50s trunk passed +1 mvnsite 1m 36s trunk passed +1 mvneclipse 0m 44s trunk passed +1 findbugs 2m 31s trunk passed +1 javadoc 1m 1s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 24s the patch passed +1 compile 2m 49s the patch passed +1 javac 2m 49s the patch passed +1 checkstyle 0m 43s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 91 unchanged - 4 fixed = 91 total (was 95) +1 mvnsite 1m 34s the patch passed +1 mvneclipse 0m 42s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 2s the patch passed -1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 3 new + 935 unchanged - 0 fixed = 938 total (was 935) +1 unit 2m 36s hadoop-yarn-common in the patch passed. +1 unit 0m 24s hadoop-yarn-server-web-proxy in the patch passed. -1 unit 37m 5s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 70m 50s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831349/YARN-4767.011.patch JIRA Issue YARN-4767 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c6bbbdff516e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 607705c Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13267/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13267/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13267/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13267/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13267/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          +1. Checking this in.

          Show
          kasha Karthik Kambatla added a comment - +1. Checking this in.
          Hide
          kasha Karthik Kambatla added a comment -

          Just committed this to trunk and branch-2. Thanks Daniel Templeton for working on this, and Vinod Kumar Vavilapalli for the reviews.

          Show
          kasha Karthik Kambatla added a comment - Just committed this to trunk and branch-2. Thanks Daniel Templeton for working on this, and Vinod Kumar Vavilapalli for the reviews.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10534 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10534/)
          YARN-4767. Network issues can cause persistent RM UI outage. (Daniel (kasha: rev 736d33cddd88a0cec925a451940b2523999a9c51)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/amfilter/AmIpFilter.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/amfilter/TestAmFilter.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/YarnWebParams.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/ErrorBlock.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RedirectionErrorPage.java
          • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRedirectionErrorPage.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebApp.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestWebAppProxyServlet.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10534 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10534/ ) YARN-4767 . Network issues can cause persistent RM UI outage. (Daniel (kasha: rev 736d33cddd88a0cec925a451940b2523999a9c51) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/amfilter/AmIpFilter.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/amfilter/TestAmFilter.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/YarnWebParams.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/ErrorBlock.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RedirectionErrorPage.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRedirectionErrorPage.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebApp.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestWebAppProxyServlet.java

            People

            • Assignee:
              templedf Daniel Templeton
              Reporter:
              templedf Daniel Templeton
            • Votes:
              2 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development