Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: scheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The Yarn Scheduler is a fertile area of interest with different implementations, e.g., Fifo, Capacity and Fair schedulers. Meanwhile, several optimizations are also made to improve scheduler performance for different scenarios and workload. Each scheduler algorithm has its own set of features, and drives scheduling decisions by many factors, such as fairness, capacity guarantee, resource availability, etc. It is very important to evaluate a scheduler algorithm very well before we deploy it in a production cluster. Unfortunately, currently it is non-trivial to evaluate a scheduling algorithm. Evaluating in a real cluster is always time and cost consuming, and it is also very hard to find a large-enough cluster. Hence, a simulator which can predict how well a scheduler algorithm for some specific workload would be quite useful.

      We want to build a Scheduler Load Simulator to simulate large-scale Yarn clusters and application loads in a single machine. This would be invaluable in furthering Yarn by providing a tool for researchers and developers to prototype new scheduler features and predict their behavior and performance with reasonable amount of confidence, there-by aiding rapid innovation.

      The simulator will exercise the real Yarn ResourceManager removing the network factor by simulating NodeManagers and ApplicationMasters via handling and dispatching NM/AMs heartbeat events from within the same JVM.

      To keep tracking of scheduler behavior and performance, a scheduler wrapper will wrap the real scheduler.

      The simulator will produce real time metrics while executing, including:

      • Resource usages for whole cluster and each queue, which can be utilized to configure cluster and queue's capacity.
      • The detailed application execution trace (recorded in relation to simulated time), which can be analyzed to understand/validate the scheduler behavior (individual jobs turn around time, throughput, fairness, capacity guarantee, etc).
      • Several key metrics of scheduler algorithm, such as time cost of each scheduler operation (allocate, handle, etc), which can be utilized by Hadoop developers to find the code spots and scalability limits.

      The simulator will provide real time charts showing the behavior of the scheduler and its performance.

      A short demo is available http://www.youtube.com/watch?v=6thLi8q0qLE, showing how to use simulator to simulate Fair Scheduler and Capacity Scheduler.

      1. YARN-1021-images.tar.gz
        698 kB
        Wei Yan
      2. YARN-1021-demo.tar.gz
        1.10 MB
        Wei Yan
      3. YARN-1021.pdf
        1.32 MB
        Wei Yan
      4. YARN-1021.patch
        1.35 MB
        Wei Yan
      5. YARN-1021.patch
        1.35 MB
        Wei Yan
      6. YARN-1021.patch
        1.35 MB
        Wei Yan
      7. YARN-1021.patch
        1.35 MB
        Wei Yan
      8. YARN-1021.patch
        1.36 MB
        Wei Yan
      9. YARN-1021.patch
        1.36 MB
        Wei Yan
      10. YARN-1021.patch
        1.35 MB
        Wei Yan
      11. YARN-1021.patch
        1.37 MB
        Wei Yan
      12. YARN-1021.patch
        1.35 MB
        Wei Yan
      13. YARN-1021.patch
        1.37 MB
        Wei Yan
      14. YARN-1021.patch
        1.38 MB
        Wei Yan
      15. YARN-1021.patch
        1.40 MB
        Wei Yan
      16. YARN-1021.patch
        1.40 MB
        Wei Yan
      17. YARN-1021.patch
        1.40 MB
        Wei Yan
      18. YARN-1021.patch
        1.40 MB
        Wei Yan

        Issue Links

          Activity

          Hide
          Bikas Saha added a comment -

          The idea and goals are very interesting. It would be great if there was a design description to initiate a discussion.

          Show
          Bikas Saha added a comment - The idea and goals are very interesting. It would be great if there was a design description to initiate a discussion.
          Hide
          Wei Yan added a comment -

          YARN-1021.pdf: simulator documentation.
          YARN-1021-demo.tar.gz: configuration (for YARN) and data used for a demo running.
          YARN-1021-images.tar.gz: images used by simulator site document.

          Show
          Wei Yan added a comment - YARN-1021 .pdf: simulator documentation. YARN-1021 -demo.tar.gz: configuration (for YARN) and data used for a demo running. YARN-1021 -images.tar.gz: images used by simulator site document.
          Hide
          Wei Yan added a comment -

          Upload a patch for the simulator. Welcome comments.

          Show
          Wei Yan added a comment - Upload a patch for the simulator. Welcome comments.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12596449/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          -1 javac. The applied patch generated 1163 javac compiler warnings (more than the trunk's current 1147 warnings).

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 28 new Findbugs (version 1.3.9) warnings.

          -1 release audit. The applied patch generated 7 release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1663//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1663//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1663//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-sls.html
          Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1663//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1663//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12596449/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. -1 javac . The applied patch generated 1163 javac compiler warnings (more than the trunk's current 1147 warnings). +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 28 new Findbugs (version 1.3.9) warnings. -1 release audit . The applied patch generated 7 release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1663//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1663//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1663//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-sls.html Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1663//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1663//console This message is automatically generated.
          Hide
          Wei Yan added a comment -

          Update a patch to remove javac & findbugs warnings. As the simulator uses jquery.js and d3.js, release audit warnings still exist as they don't use apache license.

          Show
          Wei Yan added a comment - Update a patch to remove javac & findbugs warnings. As the simulator uses jquery.js and d3.js, release audit warnings still exist as they don't use apache license.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12596714/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

          -1 release audit. The applied patch generated 4 release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1668//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1668//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1668//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12596714/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to cause Findbugs (version 1.3.9) to fail. -1 release audit . The applied patch generated 4 release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1668//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1668//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1668//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12596724/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          -1 release audit. The applied patch generated 4 release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1670//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1670//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1670//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12596724/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. -1 release audit . The applied patch generated 4 release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1670//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1670//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1670//console This message is automatically generated.
          Hide
          Wei Yan added a comment -

          New patch for excluding license checks for d3.js and jquery.js.

          Show
          Wei Yan added a comment - New patch for excluding license checks for d3.js and jquery.js.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12596733/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1672//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1672//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12596733/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1672//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1672//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          Wei, first of all, nice.

          I'm not convinced (even if I suggested that to you offline) on having dirs under share/hadoop/tools/sls/conf being 'configurable' and added to the classpath.

          Instead I would suggest the following:

          The stuff under share/hadoop/tools/sls/ should be samples, i.e.: sample-conf/ & sample-data

          The runmen2sls.sh & slsrunner.sh scripts should not add the sample-conf dir to the classpath, they should just add the JARs.

          And the documentation should state that sample-conf/ files should be copied to the hadoop conf/ directory to run the simulator.

          Show
          Alejandro Abdelnur added a comment - Wei, first of all, nice. I'm not convinced (even if I suggested that to you offline) on having dirs under share/hadoop/tools/sls/conf being 'configurable' and added to the classpath. Instead I would suggest the following: The stuff under share/hadoop/tools/sls/ should be samples, i.e.: sample-conf/ & sample-data The runmen2sls.sh & slsrunner.sh scripts should not add the sample-conf dir to the classpath, they should just add the JARs. And the documentation should state that sample-conf/ files should be copied to the hadoop conf/ directory to run the simulator.
          Hide
          Wei Yan added a comment -

          Update patch and documents according to Alejandro Abdelnur's suggestions.

          Show
          Wei Yan added a comment - Update patch and documents according to Alejandro Abdelnur 's suggestions.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12597013/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1680//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1680//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12597013/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1680//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1680//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          Patch looks good, some comments:

          Documentation:

          • 'Usage': you are assuming the user built hadoop, instead, you should explain things assuming the user is standing in the hadoop install directory, after they expand the tarball, standing where share/ and etc/.
          • 'Usage step 1': mention the sls-runner.xml has all the default values for this properties (it should if not). Also, mention that the hadoop RM/scheduler configuration will be taken from the Hadoop conf directory.
          • 'Usage step 2': use path to rumen2sls.sh from install directory.
          • 'Usage step 3': use path to slsrun.sh from install directory.

          Code:

          • AMSimulator: var definitions, one var def per line, don't comma them.
          • AMSimulator: middleStep() method, no need to use 'this.' when invoking methods.
          • AMSimulator: instead using SLF4J MessageFormatter, use JDK MessageFormat as in other places (note you'll have to change from {} to {#}

            in the templates).

          • ResourceSchedulerWrapper: instead using getRuntime() to add a shutdown hook it should use Hadoop's ShutdownHookManager

          html/simulateTraces.html should be renamed to showSimulationTrace.html. The documentation should explain clearly that a SLS run produces a simulation trace that can be seen after running the simulation.

          Show
          Alejandro Abdelnur added a comment - Patch looks good, some comments: Documentation: 'Usage': you are assuming the user built hadoop, instead, you should explain things assuming the user is standing in the hadoop install directory, after they expand the tarball, standing where share/ and etc/. 'Usage step 1': mention the sls-runner.xml has all the default values for this properties (it should if not). Also, mention that the hadoop RM/scheduler configuration will be taken from the Hadoop conf directory. 'Usage step 2': use path to rumen2sls.sh from install directory. 'Usage step 3': use path to slsrun.sh from install directory. Code: AMSimulator: var definitions, one var def per line, don't comma them. AMSimulator: middleStep() method, no need to use 'this.' when invoking methods. AMSimulator: instead using SLF4J MessageFormatter, use JDK MessageFormat as in other places (note you'll have to change from {} to {#} in the templates). ResourceSchedulerWrapper: instead using getRuntime() to add a shutdown hook it should use Hadoop's ShutdownHookManager html/simulateTraces.html should be renamed to showSimulationTrace.html. The documentation should explain clearly that a SLS run produces a simulation trace that can be seen after running the simulation.
          Hide
          Wei Yan added a comment -

          Update a patch according to Alejandro Abdelnur's comments.

          Show
          Wei Yan added a comment - Update a patch according to Alejandro Abdelnur 's comments.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12597233/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1688//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1688//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12597233/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1688//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1688//console This message is automatically generated.
          Hide
          Carlo Curino added a comment -

          We were working on a similar simulator effort. Please give me a little time to check the patch.
          I want to make sure you guys have addressed some of the corner cases we stumbled on.

          Show
          Carlo Curino added a comment - We were working on a similar simulator effort. Please give me a little time to check the patch. I want to make sure you guys have addressed some of the corner cases we stumbled on.
          Hide
          Wei Yan added a comment -

          Thanks, Carlo Curino. Looking forward to your further comments.

          Show
          Wei Yan added a comment - Thanks, Carlo Curino . Looking forward to your further comments.
          Hide
          Wei Yan added a comment -

          A new patch updates some code in the NMSimulator.

          Show
          Wei Yan added a comment - A new patch updates some code in the NMSimulator.
          Hide
          Wei Yan added a comment -

          Updates of the patch: Reduce the number of threads needed for NMSimulators. Before, each NMSimulator uses one thread (for its AsyncDispatcher). Currently removed AsyncDispatcher and the total number of threads needed only depends on the thread pool size.

          Show
          Wei Yan added a comment - Updates of the patch: Reduce the number of threads needed for NMSimulators. Before, each NMSimulator uses one thread (for its AsyncDispatcher). Currently removed AsyncDispatcher and the total number of threads needed only depends on the thread pool size.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12597774/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1705//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1705//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12597774/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1705//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1705//console This message is automatically generated.
          Hide
          Carlo Curino added a comment -

          Sorry for the delay. I went over the patch today together with Chris Douglas and here some input from the both of us.

          I generally like the effort, and the live visualization is really neat. Also making it into a completely separate tool is convenient/safe.

          The main limitations I see in this simulator are:

          • it only simulates the Scheduler code, mocking out most of the RM, and all AM and NM, communication, submissions...
          • If I am not mistaken runs at wall-clock time (not faster)
          • does not run the "monitors" which are needed for simulating preemption in the CapacityScheduler

          An alternative approach that we explored was to hijack the "Clocks" around the RM and drive them using a discrete event simulation, thus exercising more of the RM code, protocols etc... and enabling faster than wall-clock speeds (though not trivial to achieve). We have some working but not polished code in this space, which we could probably provide if you think might be integrated/leveraged.

          Ignoring alternative approaches, and broader spectrum we mentioned above, there are few issues with the current patch:

          • It should be possible to consistently replay (seed RANDOM)
          • Using Rumen reader (JobProducer, etc.) instead of parsing json directly seems cleaner. Also we have a synth load generator which we will release soon that implements the JobProducer/JobStory interface (might be nice to use that to drive your simulations)
          • LICENSE/NOTICE should be updated to include the BSD-like licenses you bring in with the new libraries
          • It seems somewhat hard to detect regressions w/ trunk since:
            • mocks away much of the AM/NM/RM
            • few unit tests
            • does not simulate important behaviors in the AM (no slow start, headroom, etc.)
            • does not exercise failures, timeouts, etc.

          Smaller issues:

          • some javadoc @param unpopulated
          • why a dependency on another metrics package, instead of Hadoop's?
          • why NodeUpdateSchedulerEventWrapper? Doesn't seem to add anything...
          • use ResourceCalculator instead of manually adjusting Resources from RR
          • initMetrics is a very large method...
          • SLSWebApp: is a wall of string appends. I am not very web savvy but I believe there should be cleaner ways to generate this. This seems hard to maintain/evolve.

          I hope this helps. I will be traveling abroad for a couple of weeks so I might be slow/unresponsive. Altogether since it is rather "on a side" I am not too concern about it, the suggestions are mostly to make sure it is really useful and that people can use it / maintain it overtime. If committed as is will do no harm, but I think it risk to be dropped in, used twice for FairScheduler work, and than loose relevance and get out of sync from trunk.

          Show
          Carlo Curino added a comment - Sorry for the delay. I went over the patch today together with Chris Douglas and here some input from the both of us. I generally like the effort, and the live visualization is really neat. Also making it into a completely separate tool is convenient/safe. The main limitations I see in this simulator are: it only simulates the Scheduler code, mocking out most of the RM, and all AM and NM, communication, submissions... If I am not mistaken runs at wall-clock time (not faster) does not run the "monitors" which are needed for simulating preemption in the CapacityScheduler An alternative approach that we explored was to hijack the "Clocks" around the RM and drive them using a discrete event simulation, thus exercising more of the RM code, protocols etc... and enabling faster than wall-clock speeds (though not trivial to achieve). We have some working but not polished code in this space, which we could probably provide if you think might be integrated/leveraged. Ignoring alternative approaches, and broader spectrum we mentioned above, there are few issues with the current patch: It should be possible to consistently replay (seed RANDOM) Using Rumen reader (JobProducer, etc.) instead of parsing json directly seems cleaner. Also we have a synth load generator which we will release soon that implements the JobProducer/JobStory interface (might be nice to use that to drive your simulations) LICENSE/NOTICE should be updated to include the BSD-like licenses you bring in with the new libraries It seems somewhat hard to detect regressions w/ trunk since: mocks away much of the AM/NM/RM few unit tests does not simulate important behaviors in the AM (no slow start, headroom, etc.) does not exercise failures, timeouts, etc. Smaller issues: some javadoc @param unpopulated why a dependency on another metrics package, instead of Hadoop's? why NodeUpdateSchedulerEventWrapper? Doesn't seem to add anything... use ResourceCalculator instead of manually adjusting Resources from RR initMetrics is a very large method... SLSWebApp: is a wall of string appends. I am not very web savvy but I believe there should be cleaner ways to generate this. This seems hard to maintain/evolve. I hope this helps. I will be traveling abroad for a couple of weeks so I might be slow/unresponsive. Altogether since it is rather "on a side" I am not too concern about it, the suggestions are mostly to make sure it is really useful and that people can use it / maintain it overtime. If committed as is will do no harm, but I think it risk to be dropped in, used twice for FairScheduler work, and than loose relevance and get out of sync from trunk.
          Hide
          Wei Yan added a comment -

          Carlo Curino. Thanks for taking the time and doing such detailed review.

          Addressing your comments:

          it only simulates the Scheduler code, mocking out most of the RM, and all AM and NM, communication, submissions...

          This is not entirely correct, the simulator exercises all RM code except for incoming/outgoing networks. The simulator dispatches/processes the events that should be received/sent via the network from/to AMs and NMs. Running all AM and NM code in the simulator would be too much overhead.

          If I am not mistaken runs at wall-clock time (not faster)

          Correct.

          Though adding a speed factor is a good idea and it would not be that difficult, though it will require some work in Hadoop (as you are suggesting later in your comments on 'hijacking Clocks': Hadoop already has a Clock interface with a SystemClock implementation, but it is hardcoded to use System.currentTimeMillis(). SystemClock should be modified to use a configurable implementation (in our speedy case, a FastSystemClock).

          The Simulator code needs to be modified to use Hadoop Clock instead of System.currentTimeMillis().

          In addition, because the simulator leverages JDK ScheduledExecutor to execute commands based on time, we need to introduce the correction there. For this Hadoop Clock interface should expose a RealTimeRatio via a new method, and the default implementation would return 1 (and the speedy one would return the corresponding factor).

          Also, we'll have to scan all RM code to ensure Clock is always used for all time base computations.

          I'll open a JIRA for this.

          does not run the "monitors" which are needed for simulating preemption in the CapacityScheduler

          If preemption is exclusively decided in the RM side, this should just work.

          If the monitors run in the NMs, then we would need to simulate this.

          Or am I missing something? If I am, I'd open a JIRA to handle this.

          It should be possible to consistently replay (seed RANDOM)

          What do you exactly mean? You can replay a load and things will work similarly but not exactly the same.

          Using Rumen reader (JobProducer, etc.) instead of parsing json directly seems cleaner. Also we have a synth load generator which we will release soon that implements the JobProducer/JobStory interface (might be nice to use that to drive your simulations)

          Sure, I'll make this changes and upload a new patch.

          LICENSE/NOTICE

          I'll update those files.

          It seems somewhat hard to detect regressions w/ trunk since: mocks away much of the AM/NM/RM

          It mocks AM/NM, it does not mock RM.

          It seems somewhat hard to detect regressions w/ trunk since: few unit tests

          I'll try to make a testcase that starts mini MR cluster, runs a job, captures the jobhistory, runs rumen on it and runs a simulation. (cannot promise, but I'll try)

          does not simulate important behaviors in the AM (no slow start, headroom, etc.)

          Yes, I'm aware of this, I'll open a new JIRA to address these things.

          does not exercise failures, timeouts, etc.

          Same as previous comment, I'll open a new JIRA to address these things.

          some javadoc @param unpopulated

          I'll take care of those

          why a dependency on another metrics package, instead of Hadoop's?

          The metrics package used by the simulator has a few features out of the box not available in Hadoop's metrics:

          • writes output files with all metrics information automatically (every # seconds) that you can use to analyze the simulation. Hadoop metrics requires more work to generate these output files.
          • these outputs also contain stddev, percentiles, min/max, etc, which are not completely implemented in Hadoop metrics.

          use ResourceCalculator instad of manually...

          I'll do that.

          initMethics is a very large method ....

          I'll break it down

          SLSWebApp: ....

          I'll make them template files

          I'll open jiras and update the patch.

          Show
          Wei Yan added a comment - Carlo Curino . Thanks for taking the time and doing such detailed review. Addressing your comments: it only simulates the Scheduler code, mocking out most of the RM, and all AM and NM, communication, submissions... This is not entirely correct, the simulator exercises all RM code except for incoming/outgoing networks. The simulator dispatches/processes the events that should be received/sent via the network from/to AMs and NMs. Running all AM and NM code in the simulator would be too much overhead. If I am not mistaken runs at wall-clock time (not faster) Correct. Though adding a speed factor is a good idea and it would not be that difficult, though it will require some work in Hadoop (as you are suggesting later in your comments on 'hijacking Clocks': Hadoop already has a Clock interface with a SystemClock implementation, but it is hardcoded to use System.currentTimeMillis(). SystemClock should be modified to use a configurable implementation (in our speedy case, a FastSystemClock). The Simulator code needs to be modified to use Hadoop Clock instead of System.currentTimeMillis(). In addition, because the simulator leverages JDK ScheduledExecutor to execute commands based on time, we need to introduce the correction there. For this Hadoop Clock interface should expose a RealTimeRatio via a new method, and the default implementation would return 1 (and the speedy one would return the corresponding factor). Also, we'll have to scan all RM code to ensure Clock is always used for all time base computations. I'll open a JIRA for this. does not run the "monitors" which are needed for simulating preemption in the CapacityScheduler If preemption is exclusively decided in the RM side, this should just work. If the monitors run in the NMs, then we would need to simulate this. Or am I missing something? If I am, I'd open a JIRA to handle this. It should be possible to consistently replay (seed RANDOM) What do you exactly mean? You can replay a load and things will work similarly but not exactly the same. Using Rumen reader (JobProducer, etc.) instead of parsing json directly seems cleaner. Also we have a synth load generator which we will release soon that implements the JobProducer/JobStory interface (might be nice to use that to drive your simulations) Sure, I'll make this changes and upload a new patch. LICENSE/NOTICE I'll update those files. It seems somewhat hard to detect regressions w/ trunk since: mocks away much of the AM/NM/RM It mocks AM/NM, it does not mock RM. It seems somewhat hard to detect regressions w/ trunk since: few unit tests I'll try to make a testcase that starts mini MR cluster, runs a job, captures the jobhistory, runs rumen on it and runs a simulation. (cannot promise, but I'll try) does not simulate important behaviors in the AM (no slow start, headroom, etc.) Yes, I'm aware of this, I'll open a new JIRA to address these things. does not exercise failures, timeouts, etc. Same as previous comment, I'll open a new JIRA to address these things. some javadoc @param unpopulated I'll take care of those why a dependency on another metrics package, instead of Hadoop's? The metrics package used by the simulator has a few features out of the box not available in Hadoop's metrics: writes output files with all metrics information automatically (every # seconds) that you can use to analyze the simulation. Hadoop metrics requires more work to generate these output files. these outputs also contain stddev, percentiles, min/max, etc, which are not completely implemented in Hadoop metrics. use ResourceCalculator instad of manually... I'll do that. initMethics is a very large method .... I'll break it down SLSWebApp: .... I'll make them template files I'll open jiras and update the patch.
          Hide
          Wei Yan added a comment -

          miss one comment.

          why NodeUpdateSchedulerEventWrapper? Doesn't seem to add anything...

          We wrap the NodeUpdateSchedulerEvent is to keep the collection of containerUpdates. If not wrap, this collection will be cleared by the scheduler (in function RMNodeImpl.pullContainerUpdates()). Also see RMNodeWrapper.

          Show
          Wei Yan added a comment - miss one comment. why NodeUpdateSchedulerEventWrapper? Doesn't seem to add anything... We wrap the NodeUpdateSchedulerEvent is to keep the collection of containerUpdates. If not wrap, this collection will be cleared by the scheduler (in function RMNodeImpl.pullContainerUpdates() ). Also see RMNodeWrapper .
          Hide
          Carlo Curino added a comment -

          I might have missed this, but can you check whether the ProportionalCapacityPreemptionPolicy (running as a monitor) gets invoked correctly when running the simulator with the CapacityScheduler (and with preemption turned on)?

          For the use of Clocks in the RM I think it was pretty consistent (if I remember correctly from our simulator attempt). Also notice that more than a faster version of time, it is important to achieve discrete event simulation, as this allows awesome debugging... just accelerating/slowing down time does not give you that. Please consider this seriously, as I think it would heavily increase the value of your simulator.

          Related to that is consistent replay (as in fully deterministic). This would be very very valuable to have. I can see this to be invaluable for debugging, testing, demonstrating features and corner cases etc.

          Simulating NM/AM is indeed costly, thought it stresses a bunch of other part of the system... pros and cons... I would suggest you to simply make sure that your architecture/design does not prevent us later on to broaden the scope of the simulation.

          Show
          Carlo Curino added a comment - I might have missed this, but can you check whether the ProportionalCapacityPreemptionPolicy (running as a monitor) gets invoked correctly when running the simulator with the CapacityScheduler (and with preemption turned on)? For the use of Clocks in the RM I think it was pretty consistent (if I remember correctly from our simulator attempt). Also notice that more than a faster version of time, it is important to achieve discrete event simulation, as this allows awesome debugging... just accelerating/slowing down time does not give you that. Please consider this seriously, as I think it would heavily increase the value of your simulator. Related to that is consistent replay (as in fully deterministic). This would be very very valuable to have. I can see this to be invaluable for debugging, testing, demonstrating features and corner cases etc. Simulating NM/AM is indeed costly, thought it stresses a bunch of other part of the system... pros and cons... I would suggest you to simply make sure that your architecture/design does not prevent us later on to broaden the scope of the simulation.
          Hide
          Wei Yan added a comment -

          Carlo Curino Thanks for your further comments.

          I might have missed this, but can you check whether the ProportionalCapacityPreemptionPolicy (running as a monitor) gets invoked correctly when running the simulator with the CapacityScheduler (and with preemption turned on)?

          Sure, I'll check that.

          For the use of Clocks in the RM I think it was pretty consistent (if I remember correctly from our simulator attempt). Also notice that more than a faster version of time, it is important to achieve discrete event simulation, as this allows awesome debugging... just accelerating/slowing down time does not give you that. Please consider this seriously, as I think it would heavily increase the value of your simulator.

          Sorry I still not understand why the simulator needs event simulation approach. Could you share more info about that?

          Related to that is consistent replay (as in fully deterministic). This would be very very valuable to have. I can see this to be invaluable for debugging, testing, demonstrating features and corner cases etc.

          Yes, consistent replay is important in some cases. I'll think about it later.

          Simulating NM/AM is indeed costly, thought it stresses a bunch of other part of the system... pros and cons... I would suggest you to simply make sure that your architecture/design does not prevent us later on to broaden the scope of the simulation.

          I agree that design of the simulator should conside future improvement. I'll update when I have some ideas.

          Show
          Wei Yan added a comment - Carlo Curino Thanks for your further comments. I might have missed this, but can you check whether the ProportionalCapacityPreemptionPolicy (running as a monitor) gets invoked correctly when running the simulator with the CapacityScheduler (and with preemption turned on)? Sure, I'll check that. For the use of Clocks in the RM I think it was pretty consistent (if I remember correctly from our simulator attempt). Also notice that more than a faster version of time, it is important to achieve discrete event simulation, as this allows awesome debugging... just accelerating/slowing down time does not give you that. Please consider this seriously, as I think it would heavily increase the value of your simulator. Sorry I still not understand why the simulator needs event simulation approach. Could you share more info about that? Related to that is consistent replay (as in fully deterministic). This would be very very valuable to have. I can see this to be invaluable for debugging, testing, demonstrating features and corner cases etc. Yes, consistent replay is important in some cases. I'll think about it later. Simulating NM/AM is indeed costly, thought it stresses a bunch of other part of the system... pros and cons... I would suggest you to simply make sure that your architecture/design does not prevent us later on to broaden the scope of the simulation. I agree that design of the simulator should conside future improvement. I'll update when I have some ideas.
          Hide
          Carlo Curino added a comment -

          The key idea behind the discrete event simulation is that you completely decouple yourself from any real-world (or slow-down/fasten) clock. This allows you to step through the execution, set debug points, and definitely get a deterministic rexec. Overall this is really nice for debugging, and as a rig for tests.

          On the other end, in this approach you tend to pick an event order and execute multiple threads in some deterministic order, which doesn't exercise all possible code interleavings. As such it is complementary to what you have, which by being less deterministic might allow you to spot timing and concurrency bugs.

          Ideally we would want a simulator that can provide both, and even more ideally a simulator that logs a non-det execution, and than it can replay it deterministically. I think this would have cut down our development of preemption related code by a good 30%.

          My 2 cents.

          (BTW I am about to travel abroad, so expect major latency and packet drops for the next 3 weeks )

          Show
          Carlo Curino added a comment - The key idea behind the discrete event simulation is that you completely decouple yourself from any real-world (or slow-down/fasten) clock. This allows you to step through the execution, set debug points, and definitely get a deterministic rexec. Overall this is really nice for debugging, and as a rig for tests. On the other end, in this approach you tend to pick an event order and execute multiple threads in some deterministic order, which doesn't exercise all possible code interleavings. As such it is complementary to what you have, which by being less deterministic might allow you to spot timing and concurrency bugs. Ideally we would want a simulator that can provide both, and even more ideally a simulator that logs a non-det execution, and than it can replay it deterministically. I think this would have cut down our development of preemption related code by a good 30%. My 2 cents. (BTW I am about to travel abroad, so expect major latency and packet drops for the next 3 weeks )
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12602846/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1903//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1903//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12602846/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1903//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1903//console This message is automatically generated.
          Hide
          Wei Yan added a comment -

          Update a new patch according to Carlo Curino's comments.
          Main updates:
          (1) Let simulator adopt rumen traces directly, using the Rumen Reader (JobStory). Document also updated.
          (2) Add a testcase that starts the simulator using an example trace. Run 2 min and stop.
          (3) Fix several minors (LICENSE/NOTICE, javadoc, string appends, etc).

          For simulating slowstart, headroom and some other behaviors, opened jira YARN-1186.
          For discreate event-based simulation, opened jira YARN-1187.

          Show
          Wei Yan added a comment - Update a new patch according to Carlo Curino 's comments. Main updates: (1) Let simulator adopt rumen traces directly, using the Rumen Reader (JobStory). Document also updated. (2) Add a testcase that starts the simulator using an example trace. Run 2 min and stop. (3) Fix several minors (LICENSE/NOTICE, javadoc, string appends, etc). For simulating slowstart, headroom and some other behaviors, opened jira YARN-1186 . For discreate event-based simulation, opened jira YARN-1187 .
          Hide
          Alejandro Abdelnur added a comment -

          Wei, patch looks good, though some NITs that should be taken care of:

          • remove LICENSE.txt/NOTICE.txt (Hadoop has those at its root)
          • all the files in src/main/resources should not be there but in a src/main/sample-conf dir else the end up in the JAR and may be picked up without the user knowing
          • the src/test/data dir should be in src/main/data (as it ends up in the distro)
          • the test running the simulation runs for 2 mins, do we need 2 mins or can we make it less? like 1min or 30secs?
          • hadoop-tools-dist/pom.xml, hadoop-sls entry, should not have <version>. there should be an entry for hadoop-sls with the version in the hadoop-project/pom.xml dependencyManagement section.
          • The slsrunner should set fs.defaultFS=file:/// in the conf used to start the RM, else the RM will attempt to connect to HDFS.
          Show
          Alejandro Abdelnur added a comment - Wei, patch looks good, though some NITs that should be taken care of: remove LICENSE.txt/NOTICE.txt (Hadoop has those at its root) all the files in src/main/resources should not be there but in a src/main/sample-conf dir else the end up in the JAR and may be picked up without the user knowing the src/test/data dir should be in src/main/data (as it ends up in the distro) the test running the simulation runs for 2 mins, do we need 2 mins or can we make it less? like 1min or 30secs? hadoop-tools-dist/pom.xml, hadoop-sls entry, should not have <version>. there should be an entry for hadoop-sls with the version in the hadoop-project/pom.xml dependencyManagement section. The slsrunner should set fs.defaultFS= file:/// in the conf used to start the RM, else the RM will attempt to connect to HDFS.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12603882/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist:

          org.apache.hadoop.yarn.sls.TestSLSRunner

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1958//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1958//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12603882/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist: org.apache.hadoop.yarn.sls.TestSLSRunner +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1958//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1958//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12603903/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist:

          org.apache.hadoop.yarn.sls.TestSLSRunner

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1961//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1961//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12603903/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 8 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist: org.apache.hadoop.yarn.sls.TestSLSRunner +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1961//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1961//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12603917/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 11 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1962//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1962//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12603917/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 11 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1962//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1962//console This message is automatically generated.
          Hide
          Wei Yan added a comment -

          Thanks, Alejandro Abdelnur.

          Update a new patch according to Alejandro Abdelnur's latest comments.

          The simulation running time in TestSLSRunner reduced to 45 seconds.

          Show
          Wei Yan added a comment - Thanks, Alejandro Abdelnur . Update a new patch according to Alejandro Abdelnur 's latest comments. The simulation running time in TestSLSRunner reduced to 45 seconds.
          Hide
          Alejandro Abdelnur added a comment -

          Thanks Wei Yan, last set of NITs I believe:

          • I still see the issue that the simulator is taking the fs.defaultFS from the hadoop conf instead the one set to file:///
          • The NM simulator initialization is much faster now, still it takes it time to get all NM simulators running. We should print some kind of feedback to the terminal while this is happening, something like every 5 seconds "N of M NMs initialized"
          • The documentation should mention the default port of the simulator console, 10001
          • The slsrun.sh script should not reset HADOOP_OPTS (this is handy to increase heap or configure to connect a remote debugger/profiler)
          Show
          Alejandro Abdelnur added a comment - Thanks Wei Yan , last set of NITs I believe: I still see the issue that the simulator is taking the fs.defaultFS from the hadoop conf instead the one set to file:/// The NM simulator initialization is much faster now, still it takes it time to get all NM simulators running. We should print some kind of feedback to the terminal while this is happening, something like every 5 seconds "N of M NMs initialized" The documentation should mention the default port of the simulator console, 10001 The slsrun.sh script should not reset HADOOP_OPTS (this is handy to increase heap or configure to connect a remote debugger/profiler)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12604747/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 11 new or modified test files.

          -1 javac. The applied patch generated 1149 javac compiler warnings (more than the trunk's current 1145 warnings).

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1995//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1995//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1995//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12604747/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 11 new or modified test files. -1 javac . The applied patch generated 1149 javac compiler warnings (more than the trunk's current 1145 warnings). +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1995//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/1995//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1995//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12604767/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 11 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist:

          org.apache.hadoop.yarn.sls.TestSLSRunner

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1996//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1996//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12604767/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 11 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist: org.apache.hadoop.yarn.sls.TestSLSRunner +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1996//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1996//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12604801/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 11 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1998//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1998//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12604801/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 11 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1998//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1998//console This message is automatically generated.
          Hide
          Wei Yan added a comment -

          Update a new patch according to Alejandro Abdelnur's latest comments.
          And also let simulator support two types of inputs:
          (1) The rumen traces, thus users can directly deploy their rumen traces to the simulator.
          (2) The simulator itself traces (sls), which is much simpler and users can easily generate various workloads. The simulator also has a tool to help users convert rumen traces to sls traces.

          Show
          Wei Yan added a comment - Update a new patch according to Alejandro Abdelnur 's latest comments. And also let simulator support two types of inputs: (1) The rumen traces, thus users can directly deploy their rumen traces to the simulator. (2) The simulator itself traces (sls), which is much simpler and users can easily generate various workloads. The simulator also has a tool to help users convert rumen traces to sls traces.
          Hide
          Alejandro Abdelnur added a comment -

          Wei Yan, we shouldn't use /tmp as that does not get clean up by the build, instead we should use a temp subdir under target/, easily done by:

          File dir = new File("target", UUID.randomUUID());
          dir.mkdirs();
          

          And the documentation, in the appendix should have a complete/simple example of an sls JSON input file as a reference.

          Show
          Alejandro Abdelnur added a comment - Wei Yan , we shouldn't use /tmp as that does not get clean up by the build, instead we should use a temp subdir under target/, easily done by: File dir = new File( "target" , UUID.randomUUID()); dir.mkdirs(); And the documentation, in the appendix should have a complete/simple example of an sls JSON input file as a reference.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12604818/YARN-1021.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 11 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1999//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1999//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12604818/YARN-1021.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 11 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-assemblies hadoop-tools/hadoop-sls hadoop-tools/hadoop-tools-dist. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1999//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1999//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          +1

          Show
          Alejandro Abdelnur added a comment - +1
          Hide
          Alejandro Abdelnur added a comment -

          Thanks Wei, great work. Committed to trunk and branch-2.

          Show
          Alejandro Abdelnur added a comment - Thanks Wei, great work. Committed to trunk and branch-2.
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #4487 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4487/)
          YARN-1021. Yarn Scheduler Load Simulator. (ywskycn via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527059)

          • /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-sls.xml
          • /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-tools.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-project/src/site/site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-openstack
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/README
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support/findbugs-exclude.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies/sls.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/rumen2sls.sh
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap-responsive.min.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap.min.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/showSimulationTrace.html
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.info.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/track.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/RumenToSLSConverter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/AMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/MRAMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf/SLSConfiguration.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/CapacitySchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ContainerSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FairSchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FifoSchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/NodeUpdateSchedulerEventWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/TaskRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils/SLSUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web/SLSWebApp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/capacity-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler-allocation.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/log4j.properties
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/sls-runner.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/yarn-site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt/SchedulerLoadSimulator.apt.vm
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css/site.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_JVM.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_arch.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_memory.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_vcores.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_memory.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_vcores.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_running_apps_containers.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_scheduler_operation_timecost.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_job.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_queue.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/TestSLSRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler/TestTaskRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils/TestSLSUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web/TestSLSWebApp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler-allocation.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.info.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/sls-runner.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/track.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/yarn-site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-tools-dist/pom.xml
          • /hadoop/common/trunk/hadoop-tools/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #4487 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4487/ ) YARN-1021 . Yarn Scheduler Load Simulator. (ywskycn via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527059 ) /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-sls.xml /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-tools.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-project/src/site/site.xml /hadoop/common/trunk/hadoop-tools/hadoop-openstack /hadoop/common/trunk/hadoop-tools/hadoop-sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/README /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support/findbugs-exclude.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies/sls.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/rumen2sls.sh /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap-responsive.min.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap.min.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/showSimulationTrace.html /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.info.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/track.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/RumenToSLSConverter.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/AMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/MRAMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf/SLSConfiguration.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/CapacitySchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ContainerSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FairSchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FifoSchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/NodeUpdateSchedulerEventWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/TaskRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils/SLSUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web/SLSWebApp.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/capacity-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler-allocation.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/log4j.properties /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/sls-runner.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/yarn-site.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt/SchedulerLoadSimulator.apt.vm /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css/site.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_JVM.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_arch.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_memory.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_vcores.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_memory.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_vcores.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_running_apps_containers.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_scheduler_operation_timecost.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_job.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_queue.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/TestSLSRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler/TestTaskRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils/TestSLSUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web/TestSLSWebApp.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler-allocation.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.info.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/sls-runner.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/track.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/yarn-site.xml /hadoop/common/trunk/hadoop-tools/hadoop-tools-dist/pom.xml /hadoop/common/trunk/hadoop-tools/pom.xml /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #346 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/346/)
          YARN-1021. Yarn Scheduler Load Simulator. (ywskycn via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527059)

          • /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-sls.xml
          • /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-tools.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-project/src/site/site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-openstack
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/README
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support/findbugs-exclude.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies/sls.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/rumen2sls.sh
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap-responsive.min.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap.min.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/showSimulationTrace.html
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.info.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/track.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/RumenToSLSConverter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/AMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/MRAMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf/SLSConfiguration.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/CapacitySchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ContainerSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FairSchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FifoSchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/NodeUpdateSchedulerEventWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/TaskRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils/SLSUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web/SLSWebApp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/capacity-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler-allocation.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/log4j.properties
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/sls-runner.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/yarn-site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt/SchedulerLoadSimulator.apt.vm
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css/site.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_JVM.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_arch.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_memory.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_vcores.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_memory.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_vcores.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_running_apps_containers.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_scheduler_operation_timecost.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_job.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_queue.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/TestSLSRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler/TestTaskRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils/TestSLSUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web/TestSLSWebApp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler-allocation.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.info.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/sls-runner.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/track.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/yarn-site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-tools-dist/pom.xml
          • /hadoop/common/trunk/hadoop-tools/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #346 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/346/ ) YARN-1021 . Yarn Scheduler Load Simulator. (ywskycn via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527059 ) /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-sls.xml /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-tools.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-project/src/site/site.xml /hadoop/common/trunk/hadoop-tools/hadoop-openstack /hadoop/common/trunk/hadoop-tools/hadoop-sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/README /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support/findbugs-exclude.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies/sls.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/rumen2sls.sh /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap-responsive.min.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap.min.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/showSimulationTrace.html /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.info.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/track.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/RumenToSLSConverter.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/AMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/MRAMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf/SLSConfiguration.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/CapacitySchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ContainerSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FairSchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FifoSchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/NodeUpdateSchedulerEventWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/TaskRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils/SLSUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web/SLSWebApp.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/capacity-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler-allocation.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/log4j.properties /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/sls-runner.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/yarn-site.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt/SchedulerLoadSimulator.apt.vm /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css/site.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_JVM.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_arch.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_memory.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_vcores.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_memory.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_vcores.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_running_apps_containers.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_scheduler_operation_timecost.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_job.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_queue.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/TestSLSRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler/TestTaskRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils/TestSLSUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web/TestSLSWebApp.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler-allocation.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.info.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/sls-runner.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/track.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/yarn-site.xml /hadoop/common/trunk/hadoop-tools/hadoop-tools-dist/pom.xml /hadoop/common/trunk/hadoop-tools/pom.xml /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #1536 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1536/)
          YARN-1021. Yarn Scheduler Load Simulator. (ywskycn via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527059)

          • /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-sls.xml
          • /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-tools.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-project/src/site/site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-openstack
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/README
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support/findbugs-exclude.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies/sls.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/rumen2sls.sh
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap-responsive.min.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap.min.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/showSimulationTrace.html
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.info.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/track.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/RumenToSLSConverter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/AMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/MRAMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf/SLSConfiguration.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/CapacitySchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ContainerSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FairSchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FifoSchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/NodeUpdateSchedulerEventWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/TaskRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils/SLSUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web/SLSWebApp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/capacity-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler-allocation.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/log4j.properties
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/sls-runner.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/yarn-site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt/SchedulerLoadSimulator.apt.vm
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css/site.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_JVM.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_arch.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_memory.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_vcores.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_memory.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_vcores.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_running_apps_containers.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_scheduler_operation_timecost.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_job.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_queue.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/TestSLSRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler/TestTaskRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils/TestSLSUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web/TestSLSWebApp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler-allocation.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.info.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/sls-runner.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/track.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/yarn-site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-tools-dist/pom.xml
          • /hadoop/common/trunk/hadoop-tools/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1536 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1536/ ) YARN-1021 . Yarn Scheduler Load Simulator. (ywskycn via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527059 ) /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-sls.xml /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-tools.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-project/src/site/site.xml /hadoop/common/trunk/hadoop-tools/hadoop-openstack /hadoop/common/trunk/hadoop-tools/hadoop-sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/README /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support/findbugs-exclude.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies/sls.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/rumen2sls.sh /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap-responsive.min.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap.min.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/showSimulationTrace.html /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.info.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/track.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/RumenToSLSConverter.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/AMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/MRAMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf/SLSConfiguration.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/CapacitySchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ContainerSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FairSchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FifoSchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/NodeUpdateSchedulerEventWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/TaskRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils/SLSUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web/SLSWebApp.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/capacity-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler-allocation.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/log4j.properties /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/sls-runner.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/yarn-site.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt/SchedulerLoadSimulator.apt.vm /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css/site.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_JVM.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_arch.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_memory.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_vcores.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_memory.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_vcores.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_running_apps_containers.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_scheduler_operation_timecost.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_job.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_queue.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/TestSLSRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler/TestTaskRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils/TestSLSUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web/TestSLSWebApp.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler-allocation.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.info.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/sls-runner.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/track.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/yarn-site.xml /hadoop/common/trunk/hadoop-tools/hadoop-tools-dist/pom.xml /hadoop/common/trunk/hadoop-tools/pom.xml /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1562 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1562/)
          YARN-1021. Yarn Scheduler Load Simulator. (ywskycn via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527059)

          • /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-sls.xml
          • /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-tools.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-project/src/site/site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-openstack
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/README
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support/findbugs-exclude.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies/sls.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/rumen2sls.sh
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap-responsive.min.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap.min.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/showSimulationTrace.html
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.info.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/track.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/RumenToSLSConverter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/AMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/MRAMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf/SLSConfiguration.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NMSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/CapacitySchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ContainerSimulator.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FairSchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FifoSchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/NodeUpdateSchedulerEventWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/TaskRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils/SLSUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web/SLSWebApp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/capacity-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler-allocation.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/log4j.properties
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/sls-runner.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/yarn-site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt/SchedulerLoadSimulator.apt.vm
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css/site.css
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_JVM.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_arch.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_memory.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_vcores.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_memory.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_vcores.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_running_apps_containers.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_scheduler_operation_timecost.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_job.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_queue.png
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/TestSLSRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler/TestTaskRunner.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils/TestSLSUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web/TestSLSWebApp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler-allocation.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.info.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/sls-runner.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/track.html.template
          • /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/yarn-site.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-tools-dist/pom.xml
          • /hadoop/common/trunk/hadoop-tools/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1562 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1562/ ) YARN-1021 . Yarn Scheduler Load Simulator. (ywskycn via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527059 ) /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-sls.xml /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-tools.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-project/src/site/site.xml /hadoop/common/trunk/hadoop-tools/hadoop-openstack /hadoop/common/trunk/hadoop-tools/hadoop-sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/README /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support /hadoop/common/trunk/hadoop-tools/hadoop-sls/dev-support/findbugs-exclude.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/assemblies/sls.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/rumen2sls.sh /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/bin/slsrun.sh /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap-responsive.min.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/css/bootstrap.min.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/showSimulationTrace.html /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/simulate.info.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/html/track.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/RumenToSLSConverter.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/AMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/MRAMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/conf/SLSConfiguration.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NMSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/CapacitySchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ContainerSimulator.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FairSchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FifoSchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/NodeUpdateSchedulerEventWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/TaskRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/utils/SLSUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/web/SLSWebApp.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/capacity-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler-allocation.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/fair-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/log4j.properties /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/sls-runner.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/sample-conf/yarn-site.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/apt/SchedulerLoadSimulator.apt.vm /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/css/site.css /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_JVM.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_arch.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_memory.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_cluster_vcores.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_memory.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_queue_allocated_vcores.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_running_apps_containers.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_scheduler_operation_timecost.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_job.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/site/resources/images/sls_track_queue.png /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/TestSLSRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/scheduler/TestTaskRunner.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils/TestSLSUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/web/TestSLSWebApp.java /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler-allocation.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/fair-scheduler.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/simulate.info.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/sls-runner.xml /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/track.html.template /hadoop/common/trunk/hadoop-tools/hadoop-sls/src/test/resources/yarn-site.xml /hadoop/common/trunk/hadoop-tools/hadoop-tools-dist/pom.xml /hadoop/common/trunk/hadoop-tools/pom.xml /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
          Hide
          Mit Desai added a comment -

          Hey Wei, FYI, I would like to inform you that the test TestSLSRunner is failing. I have created a new JIRA for that YARN-1270

          Show
          Mit Desai added a comment - Hey Wei, FYI, I would like to inform you that the test TestSLSRunner is failing. I have created a new JIRA for that YARN-1270
          Hide
          Wei Yan added a comment -

          Thanks, Mit Desai.
          I'll look into it.

          Show
          Wei Yan added a comment - Thanks, Mit Desai . I'll look into it.
          Hide
          Carlo Curino added a comment -

          Hi Wei, it would be nice to add in the README how to use the simulator (e.g., by having a super simple rumen trace, and configs one can use right away,
          including pointers to the nice visualizations you have). I was looking at it today, and while I am sure I can figure it out digging around more, quick instructions
          will make it more likely that people pick it up.

          Show
          Carlo Curino added a comment - Hi Wei, it would be nice to add in the README how to use the simulator (e.g., by having a super simple rumen trace, and configs one can use right away, including pointers to the nice visualizations you have). I was looking at it today, and while I am sure I can figure it out digging around more, quick instructions will make it more likely that people pick it up.
          Hide
          Wei Yan added a comment -

          Carlo Curino, thanks for your comment.

          Currently we put the instructions in the .pdf document and the generated site document. We already have a simple rumen trace (hadoop-sls/src/main/data/2job2min-rumen-jh.json) and example configurations (hadoop-sls/src/main/sample-conf). I'll make this clear in the README.

          YARN-1393 is created for this issue.

          Show
          Wei Yan added a comment - Carlo Curino , thanks for your comment. Currently we put the instructions in the .pdf document and the generated site document. We already have a simple rumen trace (hadoop-sls/src/main/data/2job2min-rumen-jh.json) and example configurations (hadoop-sls/src/main/sample-conf). I'll make this clear in the README. YARN-1393 is created for this issue.
          Hide
          Qi Zhang added a comment -

          Hi @Wei Yan. I am trying to use SLS but always meet with the following exception. Can you tell me what is the reason? Thank you!

          -bash-3.2$ sudo sh share/hadoop/tools/sls/bin/slsrun.sh --input-rumen=share/hadoop/tools/sls/sample-data/2jobs2min-rumen-jh.json --output-dir=share/hadoop/tools/sls/sample_output
          log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
          log4j:WARN Please initialize the log4j system properly.
          log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
          Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /usr/local/hadoop-2.3.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
          It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
          java.lang.NullPointerException
          at org.apache.hadoop.yarn.sls.web.SLSWebApp.<init>(SLSWebApp.java:82)
          at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.initMetrics(ResourceSchedulerWrapper.java:463)
          at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.setConf(ResourceSchedulerWrapper.java:162)
          at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
          at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createScheduler(ResourceManager.java:230)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:355)
          at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:775)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:197)
          at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
          at org.apache.hadoop.yarn.sls.SLSRunner.startRM(SLSRunner.java:163)
          at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:137)
          at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:524)
          Exception in thread "pool-2-thread-72" java.lang.NullPointerException
          at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.addAMRuntime(ResourceSchedulerWrapper.java:721)
          at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.lastStep(AMSimulator.java:196)
          at org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.lastStep(MRAMSimulator.java:390)
          at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:744)
          Exception in thread "pool-2-thread-98" java.lang.NullPointerException
          at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.addAMRuntime(ResourceSchedulerWrapper.java:721)
          at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.lastStep(AMSimulator.java:196)
          at org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.lastStep(MRAMSimulator.java:390)
          at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:744)

          Show
          Qi Zhang added a comment - Hi @Wei Yan. I am trying to use SLS but always meet with the following exception. Can you tell me what is the reason? Thank you! -bash-3.2$ sudo sh share/hadoop/tools/sls/bin/slsrun.sh --input-rumen=share/hadoop/tools/sls/sample-data/2jobs2min-rumen-jh.json --output-dir=share/hadoop/tools/sls/sample_output log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /usr/local/hadoop-2.3.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now. It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'. java.lang.NullPointerException at org.apache.hadoop.yarn.sls.web.SLSWebApp.<init>(SLSWebApp.java:82) at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.initMetrics(ResourceSchedulerWrapper.java:463) at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.setConf(ResourceSchedulerWrapper.java:162) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createScheduler(ResourceManager.java:230) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:355) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:775) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:197) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.sls.SLSRunner.startRM(SLSRunner.java:163) at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:137) at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:524) Exception in thread "pool-2-thread-72" java.lang.NullPointerException at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.addAMRuntime(ResourceSchedulerWrapper.java:721) at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.lastStep(AMSimulator.java:196) at org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.lastStep(MRAMSimulator.java:390) at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Exception in thread "pool-2-thread-98" java.lang.NullPointerException at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.addAMRuntime(ResourceSchedulerWrapper.java:721) at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.lastStep(AMSimulator.java:196) at org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.lastStep(MRAMSimulator.java:390) at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744)
          Hide
          Wei Yan added a comment -

          Qi Zhang. Check the resource "simulate.info.html.template". It look the sls cannot find it.
          And step into the sls directory and try again. "cd share/hadoop/tools/sls; bin/slsrun.sh --input-rumen=sample-data/2jobs2min-rumen-jh.json --output-dir=sample_output".

          Show
          Wei Yan added a comment - Qi Zhang . Check the resource "simulate.info.html.template". It look the sls cannot find it. And step into the sls directory and try again. "cd share/hadoop/tools/sls; bin/slsrun.sh --input-rumen=sample-data/2jobs2min-rumen-jh.json --output-dir=sample_output".
          Hide
          Qi Zhang added a comment -

          Wei Yan. Thank you for your suggestion, it solves the problem!
          Actually, I tried to run the slsrun.sh from many other directories expect share/hadoop/tools/sls. I think it can be more straightforward if slsrun.sh can be executed from any path.

          Show
          Qi Zhang added a comment - Wei Yan. Thank you for your suggestion, it solves the problem! Actually, I tried to run the slsrun.sh from many other directories expect share/hadoop/tools/sls. I think it can be more straightforward if slsrun.sh can be executed from any path.
          Hide
          Haiou Fang added a comment -

          Hi, Wei Yan. It throwed the following exception when I have entered the command: " bin/slsrun.sh --input-sls=/.../hadoop-2.3.0/share/hadoop/tools/sls/sample-data/2jobs2min-rumen-jh.json --output-dir=output_exp", I really have no idea about it. Can you do me a favor ? Thank you very much.

          Exception in thread "main" java.lang.NullPointerException
          at org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromSLSTrace(SLSUtils.java:95)
          at org.apache.hadoop.yarn.sls.SLSRunner.startNM(SLSRunner.java:181)
          at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:139)
          at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:524)

          Show
          Haiou Fang added a comment - Hi, Wei Yan . It throwed the following exception when I have entered the command: " bin/slsrun.sh --input-sls=/.../hadoop-2.3.0/share/hadoop/tools/sls/sample-data/2jobs2min-rumen-jh.json --output-dir=output_exp", I really have no idea about it. Can you do me a favor ? Thank you very much. Exception in thread "main" java.lang.NullPointerException at org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromSLSTrace(SLSUtils.java:95) at org.apache.hadoop.yarn.sls.SLSRunner.startNM(SLSRunner.java:181) at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:139) at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:524)
          Hide
          Haiou Fang added a comment -

          Hi, Wei Yan I am sorry to disturb you with such question and I have run the sls successful. “bin/slsrun.sh --input-sls=sls-file/sls-jobs.json --output-dir=output_sls --nodes=sls-file/sls-nodes.json ” thank you

          Show
          Haiou Fang added a comment - Hi, Wei Yan I am sorry to disturb you with such question and I have run the sls successful. “bin/slsrun.sh --input-sls=sls-file/sls-jobs.json --output-dir=output_sls --nodes=sls-file/sls-nodes.json ” thank you
          Hide
          Wei Yan added a comment -

          Haiou Fang, Sorry for late reply. Great that your problem solved.

          Show
          Wei Yan added a comment - Haiou Fang , Sorry for late reply. Great that your problem solved.
          Hide
          Cristiana Voicu added a comment -

          Hi Wei Yan! Do you have some other sample input for sls? Where can I find a json file with multiple jobs assign to various queues? I want to do some analyses on Capacity Scheduler. On http://hadoop.apache.org/docs/r2.3.0/hadoop-sls/SchedulerLoadSimulator.html I saw some diagrams with 3 queues. Can you please attach the json file used?
          Thanks!

          Show
          Cristiana Voicu added a comment - Hi Wei Yan! Do you have some other sample input for sls? Where can I find a json file with multiple jobs assign to various queues? I want to do some analyses on Capacity Scheduler. On http://hadoop.apache.org/docs/r2.3.0/hadoop-sls/SchedulerLoadSimulator.html I saw some diagrams with 3 queues. Can you please attach the json file used? Thanks!
          Hide
          Wei Yan added a comment -

          Cristiana Voicu, the SLS directly supports rumen traces. In general, you need to have some existing workload traces (i.e., from some production clusters), and then use Rumen to generate workload traces. Then let the SLS load these traces. Or you can generate some traces randomly (random # of jobs, requests, lifetime, etc).
          Sorry that I don't have the traces used in that page right now.

          Show
          Wei Yan added a comment - Cristiana Voicu , the SLS directly supports rumen traces. In general, you need to have some existing workload traces (i.e., from some production clusters), and then use Rumen to generate workload traces. Then let the SLS load these traces. Or you can generate some traces randomly (random # of jobs, requests, lifetime, etc). Sorry that I don't have the traces used in that page right now.
          Hide
          JiankunLiu added a comment -

          Hi Wei Yan,I meet with the following exception when I use this commond.
          "bin/slsrun.sh --input-rumen=sample-data/2jobs2min-rumen-jh.json --output-dir=sample_output"
          Exception in thread "pool-3-thread-1" java.lang.NullPointerException
          at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.registerAM(AMSimulator.java:300)
          at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.firstStep(AMSimulator.java:141)
          at org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.firstStep(MRAMSimulator.java:146)
          at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:84)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:744)
          Can you do me a favor ? Thank you very much.

          Show
          JiankunLiu added a comment - Hi Wei Yan,I meet with the following exception when I use this commond. "bin/slsrun.sh --input-rumen=sample-data/2jobs2min-rumen-jh.json --output-dir=sample_output" Exception in thread "pool-3-thread-1" java.lang.NullPointerException at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.registerAM(AMSimulator.java:300) at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.firstStep(AMSimulator.java:141) at org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.firstStep(MRAMSimulator.java:146) at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:84) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Can you do me a favor ? Thank you very much.
          Hide
          JiankunLiu added a comment -

          I found that it was no exception on hadoop-2.3.0. But I meet with the following exception on hadoop-2.4.0.
          Exception in thread "pool-3-thread-1" java.lang.NullPointerException
          at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.registerAM(AMSimulator.java:300)
          at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.firstStep(AMSimulator.java:141)
          at org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.firstStep(MRAMSimulator.java:146)
          at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:84)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:744)
          what should I do?

          Show
          JiankunLiu added a comment - I found that it was no exception on hadoop-2.3.0. But I meet with the following exception on hadoop-2.4.0. Exception in thread "pool-3-thread-1" java.lang.NullPointerException at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.registerAM(AMSimulator.java:300) at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.firstStep(AMSimulator.java:141) at org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.firstStep(MRAMSimulator.java:146) at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:84) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) what should I do?
          Hide
          Wei Yan added a comment -

          JiankunLiu, yes, the simulator has exception in hadoop-2.4.0, the patch provided in YARN-1726. The coming released hadoop-2.5.0 will include that patch and the simulator should run well.

          Show
          Wei Yan added a comment - JiankunLiu , yes, the simulator has exception in hadoop-2.4.0, the patch provided in YARN-1726 . The coming released hadoop-2.5.0 will include that patch and the simulator should run well.
          Hide
          Ashok Chandrasekar added a comment -

          Hi Wei Yan, when I am running the SLS with
          bin/slsrun.sh --input-rumen=sample-data/2jobs2min-rumen-jh.json --output-dir=output/ --print-simulation
          I am getting a NullPointerException as follows:
          14/11/06 03:56:05 INFO rmnode.RMNodeImpl: a2115.smile.com:3 Node Transitioned from NEW to RUNNING
          14/11/06 03:56:05 INFO capacity.CapacityScheduler: Added node a2115.smile.com:3 clusterResource: <memory:40960, vCores:40>
          Exception in thread "main" java.lang.RuntimeException: java.lang.NullPointerException
          at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
          at org.apache.hadoop.yarn.sls.SLSRunner.startAMFromRumenTraces(SLSRunner.java:394)
          at org.apache.hadoop.yarn.sls.SLSRunner.startAM(SLSRunner.java:246)
          at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:141)
          at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:524)
          Caused by: java.lang.NullPointerException
          at java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333)
          at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988)
          at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:123)
          ... 4 more

          I am using a single node pseudo distributed setup. I have tried the SLS with both Hadoop 2.3 and 2.5, but I am still getting the same exception. Can you please help me identify the problem? Thank you very much.

          Show
          Ashok Chandrasekar added a comment - Hi Wei Yan, when I am running the SLS with bin/slsrun.sh --input-rumen=sample-data/2jobs2min-rumen-jh.json --output-dir=output/ --print-simulation I am getting a NullPointerException as follows: 14/11/06 03:56:05 INFO rmnode.RMNodeImpl: a2115.smile.com:3 Node Transitioned from NEW to RUNNING 14/11/06 03:56:05 INFO capacity.CapacityScheduler: Added node a2115.smile.com:3 clusterResource: <memory:40960, vCores:40> Exception in thread "main" java.lang.RuntimeException: java.lang.NullPointerException at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131) at org.apache.hadoop.yarn.sls.SLSRunner.startAMFromRumenTraces(SLSRunner.java:394) at org.apache.hadoop.yarn.sls.SLSRunner.startAM(SLSRunner.java:246) at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:141) at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:524) Caused by: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333) at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:123) ... 4 more I am using a single node pseudo distributed setup. I have tried the SLS with both Hadoop 2.3 and 2.5, but I am still getting the same exception. Can you please help me identify the problem? Thank you very much.
          Hide
          Wei Yan added a comment -

          Hi, Ashok Chandrasekar, you can email your configuration files and "how you start the simulator" steps to my email. I can take a look.

          Show
          Wei Yan added a comment - Hi, Ashok Chandrasekar , you can email your configuration files and "how you start the simulator" steps to my email. I can take a look.
          Hide
          Fabio C added a comment -

          Hi, I am working with sls on Hadoop 2.6.0, I really need it but I'm struggling to have it running at its best. Simulation completes on command line and I can see the log entries on the screen, but:

          1- the web interface is not working. On screen I can see this exception more or less at the beginning of the workflow:
          java.lang.NullPointerException
          at org.apache.hadoop.yarn.sls.web.SLSWebApp.<init>(SLSWebApp.java:86)
          at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.initMetrics(ResourceSchedulerWrapper.java:477)
          at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.setConf(ResourceSchedulerWrapper.java:176)
          at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
          at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createScheduler
          (ResourceManager.java:291)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit
          (ResourceManager.java:484)
          at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices
          (ResourceManager.java:989)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit
          (ResourceManager.java:255)
          at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
          at org.apache.hadoop.yarn.sls.SLSRunner.startRM(SLSRunner.java:167)
          at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:141)
          at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:528)

          Not sure which object is null, but I see the folder sls/html has the expected files.

          2- I don't get the files realtimetrack.json nor jobruntime.csv, while metrics folder is correctly populated. I see some recurring exceptions, I don't know if they are related since they don't prevent the simulation to terminate:
          java.lang.NullPointerException
          at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.addAMRuntime
          (ResourceSchedulerWrapper.java:735)
          at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.lastStep(AMSimulator.java:193)
          at org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.lastStep(MRAMSimulator.java:396)
          at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:100)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)

          and also

          Exception in thread "pool-5-thread-374" java.lang.NullPointerException
          at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:104)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)

          Any help is really appreciated

          Show
          Fabio C added a comment - Hi, I am working with sls on Hadoop 2.6.0, I really need it but I'm struggling to have it running at its best. Simulation completes on command line and I can see the log entries on the screen, but: 1- the web interface is not working. On screen I can see this exception more or less at the beginning of the workflow: java.lang.NullPointerException at org.apache.hadoop.yarn.sls.web.SLSWebApp.<init>(SLSWebApp.java:86) at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.initMetrics(ResourceSchedulerWrapper.java:477) at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.setConf(ResourceSchedulerWrapper.java:176) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createScheduler (ResourceManager.java:291) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit (ResourceManager.java:484) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices (ResourceManager.java:989) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit (ResourceManager.java:255) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.sls.SLSRunner.startRM(SLSRunner.java:167) at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:141) at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:528) Not sure which object is null, but I see the folder sls/html has the expected files. 2- I don't get the files realtimetrack.json nor jobruntime.csv, while metrics folder is correctly populated. I see some recurring exceptions, I don't know if they are related since they don't prevent the simulation to terminate: java.lang.NullPointerException at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.addAMRuntime (ResourceSchedulerWrapper.java:735) at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.lastStep(AMSimulator.java:193) at org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.lastStep(MRAMSimulator.java:396) at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) and also Exception in thread "pool-5-thread-374" java.lang.NullPointerException at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:104) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Any help is really appreciated
          Hide
          Karthik Kambatla added a comment -

          Fabio C - since this JIRA has been long committed and part of a release, do you mind filing a new JIRA with Affects Version 2.6.0, so it can be fixed. Also, if you have a patch for the issue, I ll be more than happy to help get it in.

          Show
          Karthik Kambatla added a comment - Fabio C - since this JIRA has been long committed and part of a release, do you mind filing a new JIRA with Affects Version 2.6.0, so it can be fixed. Also, if you have a patch for the issue, I ll be more than happy to help get it in.
          Hide
          Fabio C added a comment -

          The fact is I'm not even sure it's a bug (for example I also got the same mysterious exceptions of Ashok Chandrasekar, then finding out it was just a missing config file). This is why I wanted to get in touch with the author first. If you can run sls and confirm the issue I'll go for a new issue submission. On the Hadoop mailing list it looks like I'm the only one using SLS.
          Regards

          Show
          Fabio C added a comment - The fact is I'm not even sure it's a bug (for example I also got the same mysterious exceptions of Ashok Chandrasekar , then finding out it was just a missing config file). This is why I wanted to get in touch with the author first. If you can run sls and confirm the issue I'll go for a new issue submission. On the Hadoop mailing list it looks like I'm the only one using SLS. Regards
          Hide
          Wei Yan added a comment -

          Thanks for the reply, Karthik Kambatla.

          Fabio C, sorry for the late reply. I just checked hadoop-2.6.0 using the sample-conf and sample-data, it works fine.
          The first exception is because the web module not loaded yet. Just need to wait 2-3 seconds after you start the simulator. This exception would not affect the simulator running.
          For the second one, could u send you config and workload files to me? I can look into it.

          Karthik Kambatla, could u help to review YARN-1393, which provides a quick-start tutorial.

          Show
          Wei Yan added a comment - Thanks for the reply, Karthik Kambatla . Fabio C , sorry for the late reply. I just checked hadoop-2.6.0 using the sample-conf and sample-data, it works fine. The first exception is because the web module not loaded yet. Just need to wait 2-3 seconds after you start the simulator. This exception would not affect the simulator running. For the second one, could u send you config and workload files to me? I can look into it. Karthik Kambatla , could u help to review YARN-1393 , which provides a quick-start tutorial.

            People

            • Assignee:
              Wei Yan
              Reporter:
              Wei Yan
            • Votes:
              0 Vote for this issue
              Watchers:
              35 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development