Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5137

AM web UI: clicking on Map Task results in 500 error

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.7, 2.1.0-beta
    • Fix Version/s: 0.23.7, 2.1.0-beta
    • Component/s: applicationmaster
    • Labels:
      None

      Description

      Go to a running mapreduce app master web UI. Click on the job, then click on the MAP task type to bring up the list of maps, then try to click on a particular map task. It fails with a 500 error. Note this doesn't exist in 0.23.6.

      Exception in the log looks like:

      2013-04-09 13:53:01,587 DEBUG [1088374@qtp-13877033-2 - /mapreduce/task/task_1365457322543_0004_m_000000] org.apache.hadoop.yarn.webapp.GenericExceptionHandler: GOT EXCEPITION
      com.sun.jersey.api.NotFoundException: null for uri: http://host.com:38158/mapreduce/task/task_1365457322543_0004_m_000000
      at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1470)
      at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
      at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
      at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
      at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
      at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
      at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:886)
      at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
      at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
      at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
      at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
      at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      at org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.doFilter(AmIpFilter.java:123)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1069)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      ...
      ...
      ...

      1. MAPREDUCE-5137.patch
        2 kB
        Thomas Graves
      2. MAPREDUCE-5137.patch
        3 kB
        Thomas Graves

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1397 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1397/)
          MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves (Revision 1466948)

          Result = SUCCESS
          jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1466948
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1397 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1397/ ) MAPREDUCE-5137 . AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves (Revision 1466948) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1466948 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1370 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1370/)
          MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves (Revision 1466948)

          Result = FAILURE
          jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1466948
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1370 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1370/ ) MAPREDUCE-5137 . AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves (Revision 1466948) Result = FAILURE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1466948 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #579 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/579/)
          MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves (Revision 1466959)

          Result = SUCCESS
          jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1466959
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #579 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/579/ ) MAPREDUCE-5137 . AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves (Revision 1466959) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1466959 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #181 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/181/)
          MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves (Revision 1466948)

          Result = SUCCESS
          jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1466948
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #181 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/181/ ) MAPREDUCE-5137 . AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves (Revision 1466948) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1466948 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
          Hide
          Jason Lowe added a comment -

          Thanks, Tom! I committed this to trunk, branch-2, and branch-0.23.

          Show
          Jason Lowe added a comment - Thanks, Tom! I committed this to trunk, branch-2, and branch-0.23.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3594 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3594/)
          MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves (Revision 1466948)

          Result = SUCCESS
          jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1466948
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3594 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3594/ ) MAPREDUCE-5137 . AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves (Revision 1466948) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1466948 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
          Hide
          Jason Lowe added a comment -

          +1, will commit shortly.

          Show
          Jason Lowe added a comment - +1, will commit shortly.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12578226/MAPREDUCE-5137.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3518//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3518//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12578226/MAPREDUCE-5137.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3518//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3518//console This message is automatically generated.
          Hide
          Jason Lowe added a comment -

          Filed MAPREDUCE-5144 to track TestAMWebApp.testConfView's poor coverage.

          Show
          Jason Lowe added a comment - Filed MAPREDUCE-5144 to track TestAMWebApp.testConfView's poor coverage.
          Hide
          Thomas Graves added a comment -

          Thanks Jason. Here is an updated patch with the test fixed.

          Show
          Thomas Graves added a comment - Thanks Jason. Here is an updated patch with the test fixed.
          Hide
          Jason Lowe added a comment -

          I took a closer look at the other tests in TestAMWebApp. They all look like they're getting mocked data except for testConfView which cuts out early because it doesn't have a job ID. I'll file a JIRA to track improving that test.

          Show
          Jason Lowe added a comment - I took a closer look at the other tests in TestAMWebApp. They all look like they're getting mocked data except for testConfView which cuts out early because it doesn't have a job ID. I'll file a JIRA to track improving that test.
          Hide
          Jason Lowe added a comment -

          Patch change looks good, but we really should have a test for this. It turns out there is an existing test that should have caught this problem, TestAMWebApp#testTaskView, but it has a bug and doesn't really test anything. It sets up an AppContext to inject for the test, but TaskPage#render ditches early because isValidRequest returns false since app.getTask() returns null. We should fix the test to setup an App object that contains the appropriate AppContext and inject that instead. That will provide the proper AppContext and also have the appropriate task setup for the test to run properly.

          It looks like there are a lot of other tests in TestAMWebApp that are broken this way. We can fix those under another JIRA.

          Show
          Jason Lowe added a comment - Patch change looks good, but we really should have a test for this. It turns out there is an existing test that should have caught this problem, TestAMWebApp#testTaskView , but it has a bug and doesn't really test anything. It sets up an AppContext to inject for the test, but TaskPage#render ditches early because isValidRequest returns false since app.getTask() returns null. We should fix the test to setup an App object that contains the appropriate AppContext and inject that instead. That will provide the proper AppContext and also have the appropriate task setup for the test to run properly. It looks like there are a lot of other tests in TestAMWebApp that are broken this way. We can fix those under another JIRA.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12578042/MAPREDUCE-5137.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3513//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3513//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12578042/MAPREDUCE-5137.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3513//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3513//console This message is automatically generated.
          Hide
          Thomas Graves added a comment -

          Patch tested manually since its the web UI. tested on trunk, branch-2, and branch-0.23.

          Show
          Thomas Graves added a comment - Patch tested manually since its the web UI. tested on trunk, branch-2, and branch-0.23.
          Hide
          Thomas Graves added a comment -

          It looks like we aren't closing the table properly.

          Caused by: org.apache.hadoop.yarn.webapp.WebAppException: Error rendering block: nestLevel=6 expected 5
          at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:66)
          at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:74)
          at org.apache.hadoop.yarn.webapp.View.render(View.java:233)
          at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:47)
          at org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
          at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:843)
          at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:54)
          at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:80)
          at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:210)
          at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.task(AppController.java:257)

          Show
          Thomas Graves added a comment - It looks like we aren't closing the table properly. Caused by: org.apache.hadoop.yarn.webapp.WebAppException: Error rendering block: nestLevel=6 expected 5 at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:66) at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:74) at org.apache.hadoop.yarn.webapp.View.render(View.java:233) at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:47) at org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117) at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:843) at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:54) at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:80) at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:210) at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.task(AppController.java:257)

            People

            • Assignee:
              Thomas Graves
              Reporter:
              Thomas Graves
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development