Hadoop YARN
  1. Hadoop YARN
  2. YARN-22

Using URI for yarn.nodemanager log dirs fails

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.3
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
      None

      Description

      If I use URIs (eg file:///home/eli/hadoop/dirs) for yarn.nodemanager.log-dirs or yarn.nodemanager.remote-app-log-dir the container log servlet fails with an NPE (works if I remove the "file" scheme). Using a URI for yarn.nodemanager.local-dirs works.

      1. YARN-22-trunk-v5.patch
        8 kB
        Mayank Bansal
      2. MAPREDUCE-4466-trunk-v4.patch
        8 kB
        Mayank Bansal
      3. MAPREDUCE-4466-trunk-v3.patch
        8 kB
        Mayank Bansal
      4. MAPREDUCE-4466-trunk-v2.patch
        8 kB
        Mayank Bansal
      5. MAPREDUCE-4466-trunk-v1.patch
        2 kB
        Mayank Bansal

        Issue Links

          Activity

          Hide
          Eli Collins added a comment -

          The following is from a recent build of trunk:

          2012-07-20 00:26:03,024 ERROR org.apache.hadoop.yarn.webapp.Dispatcher: error handling URI: /node/containerlogs/container_1342768451944_
          0003_01_000001/eli
          java.lang.reflect.InvocationTargetException
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:150)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
          at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
          at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
          at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
          at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
          at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
          at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
          at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
          at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
          at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
          at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
          at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
          at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
          at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
          at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
          at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1015)
          at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
          at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
          at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
          at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
          at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
          at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
          at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
          at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
          at org.mortbay.jetty.Server.handle(Server.java:326)
          at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
          at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
          at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
          at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
          at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
          at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
          at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
          Caused by: java.lang.NullPointerException
          at java.util.Arrays.sort(Arrays.java:1078)
          at org.apache.hadoop.yarn.server.nodemanager.webapp.ContainerLogsPage$ContainersLogsBlock.printLogs(ContainerLogsPage.java:281)
          at org.apache.hadoop.yarn.server.nodemanager.webapp.ContainerLogsPage$ContainersLogsBlock.render(ContainerLogsPage.java:137)
          at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:64)
          at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:74)
          at org.apache.hadoop.yarn.webapp.View.render(View.java:233)
          at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:47)
          at org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
          at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:843)
          at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:54)
          at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:80)

          Show
          Eli Collins added a comment - The following is from a recent build of trunk: 2012-07-20 00:26:03,024 ERROR org.apache.hadoop.yarn.webapp.Dispatcher: error handling URI: /node/containerlogs/container_1342768451944_ 0003_01_000001/eli java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:150) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263) at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178) at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795) at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58) at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118) at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1015) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) Caused by: java.lang.NullPointerException at java.util.Arrays.sort(Arrays.java:1078) at org.apache.hadoop.yarn.server.nodemanager.webapp.ContainerLogsPage$ContainersLogsBlock.printLogs(ContainerLogsPage.java:281) at org.apache.hadoop.yarn.server.nodemanager.webapp.ContainerLogsPage$ContainersLogsBlock.render(ContainerLogsPage.java:137) at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:64) at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:74) at org.apache.hadoop.yarn.webapp.View.render(View.java:233) at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:47) at org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117) at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:843) at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:54) at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:80)
          Hide
          Mayank Bansal added a comment -

          Attaching the initial patch.

          Show
          Mayank Bansal added a comment - Attaching the initial patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12538823/MAPREDUCE-4466-trunk-v1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2694//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2694//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538823/MAPREDUCE-4466-trunk-v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2694//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2694//console This message is automatically generated.
          Hide
          Mayank Bansal added a comment -

          This change effects UI and I have tested this in my environment and it works.

          Thanks,
          Mayank

          Show
          Mayank Bansal added a comment - This change effects UI and I have tested this in my environment and it works. Thanks, Mayank
          Hide
          Siddharth Seth added a comment -

          Looks like actual log rendering will also be broken - further up in ContainerLogsPage new File(this.dirsHandler.getLogPathToRead(. Also, changing getContainerLogDirs may be a cleaner fix.

          If testNMWebServer.testNMWebApp is modified to use "file://" - it ends up creating a dir structure with "file://" being the top level directory under the current working dir. That could be modified to verify the patch.

          All access to the local-dirs and log-dirs happens via the LocalDirsHandlerService - maybe we should have this convert URIs to simple strings. file:// works in other places - since Path is used instead of File.

          Show
          Siddharth Seth added a comment - Looks like actual log rendering will also be broken - further up in ContainerLogsPage new File(this.dirsHandler.getLogPathToRead( . Also, changing getContainerLogDirs may be a cleaner fix. If testNMWebServer.testNMWebApp is modified to use "file://" - it ends up creating a dir structure with "file://" being the top level directory under the current working dir. That could be modified to verify the patch. All access to the local-dirs and log-dirs happens via the LocalDirsHandlerService - maybe we should have this convert URIs to simple strings. file:// works in other places - since Path is used instead of File .
          Hide
          Mayank Bansal added a comment -

          Incorporating sid's comments.

          Thanks,
          Mayank

          Show
          Mayank Bansal added a comment - Incorporating sid's comments. Thanks, Mayank
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539344/MAPREDUCE-4466-trunk-v2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:

          org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServer

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2713//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2713//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2713//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539344/MAPREDUCE-4466-trunk-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServer +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2713//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2713//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2713//console This message is automatically generated.
          Hide
          Mayank Bansal added a comment -

          Updating patch with log message

          Show
          Mayank Bansal added a comment - Updating patch with log message
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539345/MAPREDUCE-4466-trunk-v3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:

          org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServer

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2714//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2714//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2714//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539345/MAPREDUCE-4466-trunk-v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServer +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2714//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2714//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2714//console This message is automatically generated.
          Hide
          Siddharth Seth added a comment -

          Thanks for the updated patch Mayank. Mostly looks good.
          The findbugs warning can be avoided by catching individual exceptions instead of a generic catchAll.
          The unit test has some issues. It refers to absolute paths (file:///target/) - which will break on most systems. Also TestNMWebServers isn't the best place to test this. A simple verification of getContainerLogDirs on a path with and without "file://" should be sufficient.

          Unsetting the Fix Version - that needs to be set only after the change is committed.

          Show
          Siddharth Seth added a comment - Thanks for the updated patch Mayank. Mostly looks good. The findbugs warning can be avoided by catching individual exceptions instead of a generic catchAll. The unit test has some issues. It refers to absolute paths ( file:///target/ ) - which will break on most systems. Also TestNMWebServers isn't the best place to test this. A simple verification of getContainerLogDirs on a path with and without "file://" should be sufficient. Unsetting the Fix Version - that needs to be set only after the change is committed.
          Hide
          Mayank Bansal added a comment -

          Thanks Sid for your comments.

          Incorporated all of those.

          Thanks,
          Mayank

          Show
          Mayank Bansal added a comment - Thanks Sid for your comments. Incorporated all of those. Thanks, Mayank
          Hide
          Mayank Bansal added a comment -

          Changing the Directory structure based on YARN project. Ran FindBug in local successfully.

          Thanks,
          Mayank

          Show
          Mayank Bansal added a comment - Changing the Directory structure based on YARN project. Ran FindBug in local successfully. Thanks, Mayank
          Hide
          Siddharth Seth added a comment -

          +1. Will commit this shortly. Thanks Mayank!

          The previous test failure is fixed in the latest patch, as well as the findbugs warning.

          Show
          Siddharth Seth added a comment - +1. Will commit this shortly. Thanks Mayank! The previous test failure is fixed in the latest patch, as well as the findbugs warning.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2615 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2615/)
          YARN-22. Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal) (Revision 1375829)

          Result = SUCCESS
          sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1375829
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerLogsPage.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2615 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2615/ ) YARN-22 . Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal) (Revision 1375829) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1375829 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerLogsPage.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
          Hide
          Siddharth Seth added a comment -

          Committed to trunk, branch-2 and branch-2.1.0-alpha

          Show
          Siddharth Seth added a comment - Committed to trunk, branch-2 and branch-2.1.0-alpha
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2679 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2679/)
          YARN-22. Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal) (Revision 1375829)

          Result = SUCCESS
          sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1375829
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerLogsPage.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2679 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2679/ ) YARN-22 . Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal) (Revision 1375829) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1375829 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerLogsPage.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2645 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2645/)
          YARN-22. Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal) (Revision 1375829)

          Result = FAILURE
          sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1375829
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerLogsPage.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2645 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2645/ ) YARN-22 . Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal) (Revision 1375829) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1375829 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerLogsPage.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1142 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1142/)
          YARN-22. Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal) (Revision 1375829)

          Result = FAILURE
          sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1375829
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerLogsPage.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1142 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1142/ ) YARN-22 . Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal) (Revision 1375829) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1375829 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerLogsPage.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1174 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1174/)
          YARN-22. Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal) (Revision 1375829)

          Result = FAILURE
          sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1375829
          Files :

          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerLogsPage.java
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1174 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1174/ ) YARN-22 . Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal) (Revision 1375829) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1375829 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerLogsPage.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
          Hide
          Jason Lowe added a comment -

          Note that using a URI for the nodemanager local/log dirs isn't really fixed. While using a URI no longe crashes the rendering of the web pages, the directories themselves aren't really created as intended. As Sidd mentioned in this comment the test is using an absolute path that most users will not be able to create properly when the test runs. Yet the test passes because it's not doing what the user intended. The log directory of file:///target/TestNMWebServerLogDir ends up being transformed by the LocalDirsHandlerService into ./file:/target/TestNMWebServerLogDir rather than /target/TestNMWebServerLogDir.

          Show
          Jason Lowe added a comment - Note that using a URI for the nodemanager local/log dirs isn't really fixed. While using a URI no longe crashes the rendering of the web pages, the directories themselves aren't really created as intended. As Sidd mentioned in this comment the test is using an absolute path that most users will not be able to create properly when the test runs. Yet the test passes because it's not doing what the user intended. The log directory of file:///target/TestNMWebServerLogDir ends up being transformed by the LocalDirsHandlerService into ./ file:/target/TestNMWebServerLogDir rather than /target/TestNMWebServerLogDir.
          Hide
          Jason Lowe added a comment -

          Ah, I see this issue is already being tracked by YARN-33. I stumbled across it since YARN-106 is trying to create the missing local/log directories and ensure the permissions are properly set, and the URI used in TestContainerLogsPage is for a path that typically cannot be created. I'll fix the test case as part of YARN-106.

          Show
          Jason Lowe added a comment - Ah, I see this issue is already being tracked by YARN-33 . I stumbled across it since YARN-106 is trying to create the missing local/log directories and ensure the permissions are properly set, and the URI used in TestContainerLogsPage is for a path that typically cannot be created. I'll fix the test case as part of YARN-106 .

            People

            • Assignee:
              Mayank Bansal
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development