Hadoop YARN
  1. Hadoop YARN
  2. YARN-236

RM should point tracking URL to RM web page when app fails to start

    Details

      Description

      Similar to YARN-165, the RM should redirect the tracking URL to the specific app page on the RM web UI when the application fails to start. For example, if the AM completely fails to start due to bad AM config or bad job config like invalid queuename, then the user gets the unhelpful "The requested application exited before setting a tracking URL".

      Usually the diagnostic string on the RM app page has something useful, so we might as well point there.

        Issue Links

          Activity

          Hide
          Jason Lowe added a comment -

          Patch to have the proxy servlet fallback to the RM's webpage for the app if no tracking URL is available.

          No unit test, but I manually tested both visiting the proxy URI before the app starts running and after an app crashed on startup. In both cases the page was redirected to the RM's webpage for the app.

          Show
          Jason Lowe added a comment - Patch to have the proxy servlet fallback to the RM's webpage for the app if no tracking URL is available. No unit test, but I manually tested both visiting the proxy URI before the app starts running and after an app crashed on startup. In both cases the page was redirected to the RM's webpage for the app.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12565189/YARN-236.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/352//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/352//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565189/YARN-236.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/352//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/352//console This message is automatically generated.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          I agree that the null check can hit before the app starts where redirecting is useful.

          But for crashing AMs: Shouldn't YARN-165 have already fixed the original tracking url to point to RM web age already in case of crashing AMs? I just checked the patch and seems so.

          Show
          Vinod Kumar Vavilapalli added a comment - I agree that the null check can hit before the app starts where redirecting is useful. But for crashing AMs: Shouldn't YARN-165 have already fixed the original tracking url to point to RM web age already in case of crashing AMs? I just checked the patch and seems so.
          Hide
          Jason Lowe added a comment -

          Unfortunately YARN-165 only handled the case where the AM crashes after making it to the RUNNING state. If the AM crashes before it registers with the RM then it doesn't apply. A quick way to see this in action is either set yarn.app.mapreduce.am.command-opts to some garbage string or run a wordcount job with mapreduce.jobtracker.split.metainfo.maxsize set to 1.

          Show
          Jason Lowe added a comment - Unfortunately YARN-165 only handled the case where the AM crashes after making it to the RUNNING state. If the AM crashes before it registers with the RM then it doesn't apply. A quick way to see this in action is either set yarn.app.mapreduce.am.command-opts to some garbage string or run a wordcount job with mapreduce.jobtracker.split.metainfo.maxsize set to 1.
          Hide
          Jonathan Eagles added a comment -

          +1. Verified this condition still exists and patch does redirect to the RM app webpage. Thanks, Jason.

          Show
          Jonathan Eagles added a comment - +1. Verified this condition still exists and patch does redirect to the RM app webpage. Thanks, Jason.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3368 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3368/)
          YARN-236. RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) (Revision 1448406)

          Result = SUCCESS
          jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448406
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3368 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3368/ ) YARN-236 . RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) (Revision 1448406) Result = SUCCESS jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448406 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #134 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/134/)
          YARN-236. RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) (Revision 1448406)

          Result = SUCCESS
          jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448406
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #134 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/134/ ) YARN-236 . RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) (Revision 1448406) Result = SUCCESS jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448406 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #532 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/532/)
          YARN-236. RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) (Revision 1448411)

          Result = SUCCESS
          jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448411
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #532 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/532/ ) YARN-236 . RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) (Revision 1448411) Result = SUCCESS jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448411 Files : /hadoop/common/branches/branch-0.23/hadoop-yarn-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1323 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1323/)
          YARN-236. RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) (Revision 1448406)

          Result = SUCCESS
          jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448406
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1323 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1323/ ) YARN-236 . RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) (Revision 1448406) Result = SUCCESS jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448406 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1351 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1351/)
          YARN-236. RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) (Revision 1448406)

          Result = FAILURE
          jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448406
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1351 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1351/ ) YARN-236 . RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles) (Revision 1448406) Result = FAILURE jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448406 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java

            People

            • Assignee:
              Jason Lowe
              Reporter:
              Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development