Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Configure Fair scheduler in yarn site
      Start SLS check cluster apps page

      15/12/27 07:40:22 INFO scheduler.SchedulerNode: Assigned container container_1451182067412_0002_01_000258 of capacity <memory:1024, vCores:1> on host a2117.smile.com:2, which has 10 containers, <memory:10240, vCores:10> used and <memory:0, vCores:0> available after allocation
      15/12/27 07:40:22 ERROR webapp.Dispatcher: error handling URI: /cluster/apps
      java.lang.NullPointerException
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getApplicationAttempt(AbstractYarnScheduler.java:299)
      	at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppAttemptBlock.getBlacklistedNodes(RMAppAttemptBlock.java:260)
      	at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppsBlock.renderData(RMAppsBlock.java:99)
      	at org.apache.hadoop.yarn.server.webapp.AppsBlock.render(AppsBlock.java:140)
      	at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
      	at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
      	at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
      	at org.apache.hadoop.yarn.webapp.view.HtmlBlock$Block.subView(HtmlBlock.java:43)
      	at org.apache.hadoop.yarn.webapp.hamlet.Hamlet._(Hamlet.java:30347)
      	at org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlockWithMetrics.render(AppsBlockWithMetrics.java:30)
      	at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
      	at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
      	at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
      	at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
      	at org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
      	at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:845)
      	at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
      	at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
      	at org.apache.hadoop.yarn.webapp.Dispatcher.render(Dispatcher.java:197)
      	at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:156)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
      	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
      	at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
      
      

      Confgure Capacity scheduler and SLS start up

      java.lang.NullPointerException
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:1038)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:1319)
      	at org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler.handle(SLSCapacityScheduler.java:252)
      	at org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler.handle(SLSCapacityScheduler.java:82)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:691)
      	at java.lang.Thread.run(Thread.java:745)
      
      

      SLS failed to start

      1. 0001-YARN-4510.patch
        2 kB
        Bibin A Chundatt
      2. 0002-YARN-4510.patch
        2 kB
        Bibin A Chundatt

        Activity

        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9041 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9041/)
        YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via (wangda: rev a9594c61bb8ca9e61e367988d3012a4615026090)

        • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
        • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9041 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9041/ ) YARN-4510 . Fix SLS startup failure caused by NPE. (Bibin A Chundatt via (wangda: rev a9594c61bb8ca9e61e367988d3012a4615026090) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java hadoop-yarn-project/CHANGES.txt
        Hide
        leftnoteasy Wangda Tan added a comment -

        Committed to trunk/branch-2, thanks Bibin A Chundatt!

        Show
        leftnoteasy Wangda Tan added a comment - Committed to trunk/branch-2, thanks Bibin A Chundatt !
        Hide
        leftnoteasy Wangda Tan added a comment -

        Looks good, +1, thanks Bibin A Chundatt.

        Show
        leftnoteasy Wangda Tan added a comment - Looks good, +1, thanks Bibin A Chundatt .
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Attached patch after checkstyle patch

        Show
        bibinchundatt Bibin A Chundatt added a comment - Attached patch after checkstyle patch
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 29s trunk passed
        +1 compile 0m 13s trunk passed with JDK v1.8.0_66
        +1 compile 0m 16s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 9s trunk passed
        +1 mvnsite 0m 21s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 33s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 17s the patch passed
        +1 compile 0m 11s the patch passed with JDK v1.8.0_66
        +1 javac 0m 11s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.7.0_91
        +1 javac 0m 13s the patch passed
        +1 checkstyle 0m 9s the patch passed
        +1 mvnsite 0m 19s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 36s the patch passed
        +1 javadoc 0m 10s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91
        +1 unit 0m 48s hadoop-sls in the patch passed with JDK v1.8.0_66.
        +1 unit 0m 51s hadoop-sls in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 16s Patch does not generate ASF License warnings.
        15m 1s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779947/0002-YARN-4510.patch
        JIRA Issue YARN-4510
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux a0935db85b05 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ad997fa
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10123/testReport/
        modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10123/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 29s trunk passed +1 compile 0m 13s trunk passed with JDK v1.8.0_66 +1 compile 0m 16s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 9s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 33s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 17s the patch passed +1 compile 0m 11s the patch passed with JDK v1.8.0_66 +1 javac 0m 11s the patch passed +1 compile 0m 13s the patch passed with JDK v1.7.0_91 +1 javac 0m 13s the patch passed +1 checkstyle 0m 9s the patch passed +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 36s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91 +1 unit 0m 48s hadoop-sls in the patch passed with JDK v1.8.0_66. +1 unit 0m 51s hadoop-sls in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 16s Patch does not generate ASF License warnings. 15m 1s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779947/0002-YARN-4510.patch JIRA Issue YARN-4510 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a0935db85b05 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ad997fa Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10123/testReport/ modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10123/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 45s trunk passed
        +1 compile 0m 13s trunk passed with JDK v1.8.0_66
        +1 compile 0m 16s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 9s trunk passed
        +1 mvnsite 0m 22s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 32s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 16s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 16s the patch passed
        +1 compile 0m 10s the patch passed with JDK v1.8.0_66
        +1 javac 0m 10s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.7.0_91
        +1 javac 0m 13s the patch passed
        -1 checkstyle 0m 9s Patch generated 2 new checkstyle issues in hadoop-tools/hadoop-sls (total was 41, now 43).
        +1 mvnsite 0m 19s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 35s the patch passed
        +1 javadoc 0m 10s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 14s the patch passed with JDK v1.7.0_91
        +1 unit 0m 48s hadoop-sls in the patch passed with JDK v1.8.0_66.
        +1 unit 0m 52s hadoop-sls in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        28m 43s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779938/0001-YARN-4510.patch
        JIRA Issue YARN-4510
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux bcc86c2b60a3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ad997fa
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10122/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-sls.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10122/testReport/
        modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls
        Max memory used 75MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10122/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 45s trunk passed +1 compile 0m 13s trunk passed with JDK v1.8.0_66 +1 compile 0m 16s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 9s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 32s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 16s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 16s the patch passed +1 compile 0m 10s the patch passed with JDK v1.8.0_66 +1 javac 0m 10s the patch passed +1 compile 0m 13s the patch passed with JDK v1.7.0_91 +1 javac 0m 13s the patch passed -1 checkstyle 0m 9s Patch generated 2 new checkstyle issues in hadoop-tools/hadoop-sls (total was 41, now 43). +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 35s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 14s the patch passed with JDK v1.7.0_91 +1 unit 0m 48s hadoop-sls in the patch passed with JDK v1.8.0_66. +1 unit 0m 52s hadoop-sls in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 28m 43s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779938/0001-YARN-4510.patch JIRA Issue YARN-4510 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bcc86c2b60a3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ad997fa Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10122/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-sls.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10122/testReport/ modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10122/console This message was automatically generated.
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Attaching patch for fixing start up failure of SLS and application Page. Startup issue looks related to YARN-1651 .

        Show
        bibinchundatt Bibin A Chundatt added a comment - Attaching patch for fixing start up failure of SLS and application Page. Startup issue looks related to YARN-1651 .

          People

          • Assignee:
            bibinchundatt Bibin A Chundatt
            Reporter:
            bibinchundatt Bibin A Chundatt
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development