Details

    • Hadoop Flags:
      Reviewed

      Description

      java.lang.NullPointerException
      	at org.apache.hadoop.yarn.sls.web.SLSWebApp.<init>(SLSWebApp.java:88)
      	at org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler.initMetrics(SLSCapacityScheduler.java:459)
      	at org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler.setConf(SLSCapacityScheduler.java:153)
      	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
      

      CLASSPATH for html resource is not configured properly.

      DEBUG: Injecting share/hadoop/tools/sls/html into CLASSPATH
      DEBUG: Rejected CLASSPATH: share/hadoop/tools/sls/html (does not exist)
      

      This issue can be reproduced when doing according to the documentation instruction.
      http://hadoop.apache.org/docs/current/hadoop-sls/SchedulerLoadSimulator.html

      $ cd $HADOOP_ROOT/share/hadoop/tools/sls
      $ bin/slsrun.sh
        --input-rumen |--input-sls=<TRACE_FILE1,TRACE_FILE2,...>
        --output-dir=<SLS_SIMULATION_OUTPUT_DIRECTORY> [--nodes=<SLS_NODES_FILE>]
          [--track-jobs=<JOBID1,JOBID2,...>] [--print-simulation]
      
      1. YARN-5706.01.patch
        0.7 kB
        Kai Sasaki
      2. YARN-5706.02.patch
        0.7 kB
        Kai Sasaki
      3. YARN-5706.03.patch
        0.7 kB
        Kai Sasaki

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        0 shelldocs 0m 4s Shelldocs was not available.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 39s trunk passed
        +1 mvnsite 0m 29s trunk passed
        +1 mvnsite 0m 21s the patch passed
        +1 shellcheck 0m 12s The patch generated 0 new + 74 unchanged - 1 fixed = 74 total (was 75)
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 unit 0m 16s hadoop-sls in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        9m 49s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831527/YARN-5706.01.patch
        JIRA Issue YARN-5706
        Optional Tests asflicense mvnsite unit shellcheck shelldocs
        uname Linux 0d5c69c00722 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ef7f06f
        shellcheck v0.4.4
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13275/testReport/
        modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13275/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. 0 shelldocs 0m 4s Shelldocs was not available. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 39s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvnsite 0m 21s the patch passed +1 shellcheck 0m 12s The patch generated 0 new + 74 unchanged - 1 fixed = 74 total (was 75) +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 0m 16s hadoop-sls in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 9m 49s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831527/YARN-5706.01.patch JIRA Issue YARN-5706 Optional Tests asflicense mvnsite unit shellcheck shelldocs uname Linux 0d5c69c00722 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ef7f06f shellcheck v0.4.4 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13275/testReport/ modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls Console output https://builds.apache.org/job/PreCommit-YARN-Build/13275/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        aw Allen Wittenauer added a comment - - edited

        It should be

        ${HADOOP_TOOLS_HOME}/${HADOOP_TOOLS_DIR}/sls/html 
        
        Show
        aw Allen Wittenauer added a comment - - edited It should be ${HADOOP_TOOLS_HOME}/${HADOOP_TOOLS_DIR}/sls/html
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        0 shelldocs 0m 6s Shelldocs was not available.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 6m 58s trunk passed
        +1 mvnsite 0m 25s trunk passed
        +1 mvnsite 0m 20s the patch passed
        +1 shellcheck 0m 12s The patch generated 0 new + 74 unchanged - 1 fixed = 74 total (was 75)
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 unit 0m 15s hadoop-sls in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        9m 5s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832127/YARN-5706.02.patch
        JIRA Issue YARN-5706
        Optional Tests asflicense mvnsite unit shellcheck shelldocs
        uname Linux 8b7294c69c38 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ebd4f39
        shellcheck v0.4.4
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13316/testReport/
        modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13316/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. 0 shelldocs 0m 6s Shelldocs was not available. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 58s trunk passed +1 mvnsite 0m 25s trunk passed +1 mvnsite 0m 20s the patch passed +1 shellcheck 0m 12s The patch generated 0 new + 74 unchanged - 1 fixed = 74 total (was 75) +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 0m 15s hadoop-sls in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 9m 5s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832127/YARN-5706.02.patch JIRA Issue YARN-5706 Optional Tests asflicense mvnsite unit shellcheck shelldocs uname Linux 8b7294c69c38 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ebd4f39 shellcheck v0.4.4 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13316/testReport/ modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls Console output https://builds.apache.org/job/PreCommit-YARN-Build/13316/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        lewuathe Kai Sasaki added a comment -

        Allen Wittenauer Thanks. I updated to use HADOOP_TOOLS_DIR. Could you review again?

        Show
        lewuathe Kai Sasaki added a comment - Allen Wittenauer Thanks. I updated to use HADOOP_TOOLS_DIR . Could you review again?
        Hide
        leftnoteasy Wangda Tan added a comment - - edited

        +1. thanks Kai Sasaki, will commit tomorrow if no opposite opinions.

        Show
        leftnoteasy Wangda Tan added a comment - - edited +1. thanks Kai Sasaki , will commit tomorrow if no opposite opinions.
        Hide
        lewuathe Kai Sasaki added a comment -

        Wangda Tan Thanks for review! Have you already merged it? Or should I rebase with trunk?

        Show
        lewuathe Kai Sasaki added a comment - Wangda Tan Thanks for review! Have you already merged it? Or should I rebase with trunk?
        Hide
        lewuathe Kai Sasaki added a comment -

        I rebased on trunk.

        Show
        lewuathe Kai Sasaki added a comment - I rebased on trunk.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 13m 50s trunk passed
        +1 mvnsite 0m 23s trunk passed
        +1 mvnsite 0m 18s the patch passed
        +1 shellcheck 0m 11s The patch generated 0 new + 116 unchanged - 1 fixed = 116 total (was 117)
        +1 shelldocs 0m 12s There were no new shelldocs issues.
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 unit 0m 16s hadoop-sls in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        16m 9s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-5706
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843759/YARN-5706.03.patch
        Optional Tests asflicense mvnsite unit shellcheck shelldocs
        uname Linux fe81925ce735 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / fcbe152
        shellcheck v0.4.5
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14358/testReport/
        modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14358/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 13m 50s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvnsite 0m 18s the patch passed +1 shellcheck 0m 11s The patch generated 0 new + 116 unchanged - 1 fixed = 116 total (was 117) +1 shelldocs 0m 12s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 0m 16s hadoop-sls in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 16m 9s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-5706 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843759/YARN-5706.03.patch Optional Tests asflicense mvnsite unit shellcheck shelldocs uname Linux fe81925ce735 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fcbe152 shellcheck v0.4.5 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14358/testReport/ modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls Console output https://builds.apache.org/job/PreCommit-YARN-Build/14358/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Kai Sasaki, apologize I forgot to merge it last time. Just committed to trunk, thanks!

        And do we need this fix in branch-2/branch-2.8?

        Show
        leftnoteasy Wangda Tan added a comment - Kai Sasaki , apologize I forgot to merge it last time. Just committed to trunk, thanks! And do we need this fix in branch-2/branch-2.8?
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11017 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11017/)
        YARN-5706. Fail to launch SLSRunner due to NPE. (Kai Sasaki via wangda) (wangda: rev 3bcfe3a9de2cde67092d0eb13f94d361feab6ec8)

        • (edit) hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11017 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11017/ ) YARN-5706 . Fail to launch SLSRunner due to NPE. (Kai Sasaki via wangda) (wangda: rev 3bcfe3a9de2cde67092d0eb13f94d361feab6ec8) (edit) hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh
        Hide
        lewuathe Kai Sasaki added a comment -

        Wangda Tan Thanks!

        The change was introduced in this commit. It seems to be merged into trunk and release-3.0.0-alpha1-RC0.
        So I think it isn't necessary to backport to branch-2/branch-2.8.

        Show
        lewuathe Kai Sasaki added a comment - Wangda Tan Thanks! The change was introduced in this commit . It seems to be merged into trunk and release-3.0.0-alpha1-RC0 . So I think it isn't necessary to backport to branch-2/branch-2.8.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Kai Sasaki, got it, thanks for the additional notes!

        Show
        leftnoteasy Wangda Tan added a comment - Kai Sasaki , got it, thanks for the additional notes!

          People

          • Assignee:
            lewuathe Kai Sasaki
            Reporter:
            lewuathe Kai Sasaki
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development