Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4109

Exception on RM scheduler page loading with labels

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Configure node label and load scheduler Page
      On each reload of the page the below exception gets thrown in logs

      2015-09-03 11:27:08,544 ERROR org.apache.hadoop.yarn.webapp.Dispatcher: error handling URI: /cluster/scheduler
      java.lang.reflect.InvocationTargetException
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:153)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
      	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
      	at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
      	at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
      	at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
      	at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:139)
      	at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
      	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
      	at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
      	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
      	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      	at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
      	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:663)
      	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
      	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:615)
      	at org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter.doFilter(RMAuthenticationFilter.java:82)
      	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      	at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1211)
      	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      	at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
      	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      	at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
      	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
      	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
      	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
      	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
      	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
      	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
      	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
      	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
      	at org.mortbay.jetty.Server.handle(Server.java:326)
      	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
      	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
      	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
      	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
      	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
      	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
      	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
      Caused by: org.apache.hadoop.yarn.webapp.WebAppException: Error rendering block: nestLevel=10 expected 5
      	at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:71)
      	at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
      	at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
      	at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
      	at org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
      	at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:845)
      	at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:56)
      	at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
      	at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
      	at org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.scheduler(RmController.java:82)
      	... 47 more
      
      
      1. YARN-4109_1.patch
        1 kB
        Mohammad Shahid Khan

        Activity

        Hide
        mohdshahidkhan Mohammad Shahid Khan added a comment -

        This issue is happening when the node-labels is enabled, one of the tag is not closed properly.
        Submitting the patch for the same.

        Show
        mohdshahidkhan Mohammad Shahid Khan added a comment - This issue is happening when the node-labels is enabled, one of the tag is not closed properly. Submitting the patch for the same.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        +1 lgtm, pending jenkins

        Show
        rohithsharma Rohith Sharma K S added a comment - +1 lgtm, pending jenkins
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 48s trunk passed
        +1 compile 0m 28s trunk passed with JDK v1.8.0_66
        +1 compile 0m 30s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 12s trunk passed
        +1 mvnsite 0m 37s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 11s trunk passed
        +1 javadoc 0m 21s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 27s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 33s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.8.0_66
        +1 javac 0m 26s the patch passed
        +1 compile 0m 30s the patch passed with JDK v1.7.0_91
        +1 javac 0m 30s the patch passed
        +1 checkstyle 0m 12s the patch passed
        +1 mvnsite 0m 36s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 18s the patch passed
        +1 javadoc 0m 20s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 27s the patch passed with JDK v1.7.0_91
        -1 unit 58m 51s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
        -1 unit 60m 6s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        136m 47s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
        JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775857/YARN-4109_1.patch
        JIRA Issue YARN-4109
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b06ab406fad8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 70d6f20
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10071/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10071/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10071/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10071/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10071/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Max memory used 75MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10071/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 48s trunk passed +1 compile 0m 28s trunk passed with JDK v1.8.0_66 +1 compile 0m 30s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 12s trunk passed +1 mvnsite 0m 37s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 11s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 33s the patch passed +1 compile 0m 26s the patch passed with JDK v1.8.0_66 +1 javac 0m 26s the patch passed +1 compile 0m 30s the patch passed with JDK v1.7.0_91 +1 javac 0m 30s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 36s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 18s the patch passed +1 javadoc 0m 20s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 27s the patch passed with JDK v1.7.0_91 -1 unit 58m 51s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 60m 6s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 136m 47s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775857/YARN-4109_1.patch JIRA Issue YARN-4109 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b06ab406fad8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 70d6f20 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/10071/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10071/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10071/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10071/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10071/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10071/console This message was automatically generated.
        Hide
        mohdshahidkhan Mohammad Shahid Khan added a comment -

        UT Failure are unrelated to current patch.
        UT addition/modification not required only small UI change.

        Show
        mohdshahidkhan Mohammad Shahid Khan added a comment - UT Failure are unrelated to current patch. UT addition/modification not required only small UI change.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Committed to trunk/branch-2. thanks Mohammad Shahid Khan for the patch!!

        Show
        rohithsharma Rohith Sharma K S added a comment - Committed to trunk/branch-2. thanks Mohammad Shahid Khan for the patch!!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9017 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9017/)
        YARN-4109. Exception on RM scheduler page loading with labels. (Mohammad (rohithsharmaks: rev 8c180a13c82ab9d60f595e6942e35d51024dab53)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9017 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9017/ ) YARN-4109 . Exception on RM scheduler page loading with labels. (Mohammad (rohithsharmaks: rev 8c180a13c82ab9d60f595e6942e35d51024dab53) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java hadoop-yarn-project/CHANGES.txt
        Hide
        leftnoteasy Wangda Tan added a comment -

        Hi Rohith Sharma K S/Mohammad Shahid Khan
        Do you think is better to add this to branch-2.8?

        Show
        leftnoteasy Wangda Tan added a comment - Hi Rohith Sharma K S / Mohammad Shahid Khan Do you think is better to add this to branch-2.8?
        Hide
        sunilg Sunil G added a comment -

        +1. Yes this will be helpful in 2.8 release.

        Show
        sunilg Sunil G added a comment - +1. Yes this will be helpful in 2.8 release.
        Hide
        jianhe Jian He added a comment -

        Rohith Sharma K S, seems this is not yet in branch-2.8. should we get it in ?

        Show
        jianhe Jian He added a comment - Rohith Sharma K S , seems this is not yet in branch-2.8. should we get it in ?
        Hide
        sunilg Sunil G added a comment -

        Yes correct. This can be back backported to 2.8. Mohammad Shahid Khan / Rohith Sharma K S, could you pls share a patch.

        Show
        sunilg Sunil G added a comment - Yes correct. This can be back backported to 2.8. Mohammad Shahid Khan / Rohith Sharma K S , could you pls share a patch.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Apologies for missing earlier comments. Let me back port this to branch-2.8

        Show
        rohithsharma Rohith Sharma K S added a comment - Apologies for missing earlier comments. Let me back port this to branch-2.8
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Back ported to branch-2.8!! And changed the fix version/s as to 2.8.

        Show
        rohithsharma Rohith Sharma K S added a comment - Back ported to branch-2.8!! And changed the fix version/s as to 2.8.
        Hide
        sunilg Sunil G added a comment -

        I think we need to take this for branch-2.7 also. Vinod Kumar Vavilapalli, I noticed that this error is shown in log in the 2.7.3 release.

        Since this is not critical, i feel we can move to 2.7.4. Vinod Kumar Vavilapalli and Rohith Sharma K S sounds good?

        Show
        sunilg Sunil G added a comment - I think we need to take this for branch-2.7 also. Vinod Kumar Vavilapalli , I noticed that this error is shown in log in the 2.7.3 release. Since this is not critical, i feel we can move to 2.7.4. Vinod Kumar Vavilapalli and Rohith Sharma K S sounds good?
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        I do not see any functional impact from this other than an exception in RM logs. I think this can be moved to 2.7.4 given other folks are also fine.

        Show
        rohithsharma Rohith Sharma K S added a comment - I do not see any functional impact from this other than an exception in RM logs. I think this can be moved to 2.7.4 given other folks are also fine.
        Hide
        jhung Jonathan Hung added a comment -

        Konstantin Shvachko this applies cleanly to branch-2.7. Can we backport it there? Thanks!

        Show
        jhung Jonathan Hung added a comment - Konstantin Shvachko this applies cleanly to branch-2.7. Can we backport it there? Thanks!
        Hide
        shv Konstantin Shvachko added a comment -

        Just committed this to branch-2.7.

        Show
        shv Konstantin Shvachko added a comment - Just committed this to branch-2.7.

          People

          • Assignee:
            mohdshahidkhan Mohammad Shahid Khan
            Reporter:
            bibinchundatt Bibin A Chundatt
          • Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development