Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3050

YarnScheduler needs to expose Resource Usage Information

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 0.24.0
    • Fix Version/s: 0.23.0
    • Component/s: mrv2, resourcemanager
    • Labels:
      None

      Description

      Before the recent refactor The nodes had information in them about how much resources they were using. This information is not hidden inside SchedulerNode. Similarly resource usage information about an application, or in aggregate is only available through the Scheduler and there is not interface to pull it out.

      We need to expose APIs to get Resource and Container information from the scheduler, in aggregate across the entire cluster, per application, per node, and ideally also per queue if applicable (although there are no JIRAs I am aware of that need this right now).

      1. MR-3050.txt
        90 kB
        Robert Joseph Evans
      2. MR-3050.txt
        91 kB
        Robert Joseph Evans
      3. MR-3050.txt
        91 kB
        Robert Joseph Evans
      4. MR-3050.txt
        91 kB
        Robert Joseph Evans
      5. MR-3050.txt
        27 kB
        Robert Joseph Evans
      6. MR-3050.txt
        26 kB
        Robert Joseph Evans
      7. MR-3050.txt
        25 kB
        Robert Joseph Evans

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #26 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/26/)
          Merge -r 1177858:1177859 from trunk to branch-0.23 to fix MAPREDUCE-3050.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177861
          Files :

          • /hadoop/common/branches/branch-0.23
          • /hadoop/common/branches/branch-0.23/hadoop-common-project
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf/capacity-scheduler.xml.template
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/capacity-scheduler
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/dynamic-scheduler
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/fairscheduler
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/streaming
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/test/MapredTestDriver.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #26 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/26/ ) Merge -r 1177858:1177859 from trunk to branch-0.23 to fix MAPREDUCE-3050 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177861 Files : /hadoop/common/branches/branch-0.23 /hadoop/common/branches/branch-0.23/hadoop-common-project /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf/capacity-scheduler.xml.template /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++ /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/capacity-scheduler /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/dynamic-scheduler /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/fairscheduler /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/streaming /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/test/MapredTestDriver.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #847 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/847/)
          MAPREDUCE-3050. Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #847 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/847/ ) MAPREDUCE-3050 . Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #33 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/33/)
          Merge -r 1177858:1177859 from trunk to branch-0.23 to fix MAPREDUCE-3050.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177861
          Files :

          • /hadoop/common/branches/branch-0.23
          • /hadoop/common/branches/branch-0.23/hadoop-common-project
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf/capacity-scheduler.xml.template
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/capacity-scheduler
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/dynamic-scheduler
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/fairscheduler
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/streaming
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/test/MapredTestDriver.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #33 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/33/ ) Merge -r 1177858:1177859 from trunk to branch-0.23 to fix MAPREDUCE-3050 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177861 Files : /hadoop/common/branches/branch-0.23 /hadoop/common/branches/branch-0.23/hadoop-common-project /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf/capacity-scheduler.xml.template /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++ /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/capacity-scheduler /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/dynamic-scheduler /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/fairscheduler /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/streaming /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/test/MapredTestDriver.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #817 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/817/)
          MAPREDUCE-3050. Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #817 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/817/ ) MAPREDUCE-3050 . Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1014 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1014/)
          MAPREDUCE-3050. Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1014 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1014/ ) MAPREDUCE-3050 . Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #994 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/994/)
          MAPREDUCE-3050. Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #994 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/994/ ) MAPREDUCE-3050 . Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1072 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1072/)
          MAPREDUCE-3050. Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1072 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1072/ ) MAPREDUCE-3050 . Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
          Hide
          Arun C Murthy added a comment -

          I just committed this. Thanks Robert!

          Show
          Arun C Murthy added a comment - I just committed this. Thanks Robert!
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12497212/MR-3050.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/916//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/916//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497212/MR-3050.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/916//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/916//console This message is automatically generated.
          Hide
          Robert Joseph Evans added a comment -

          Refactored SchedulerNodeReport and SchedulerAppReport to not be a part of ScheduelrNode and SchedulerApp respectively.

          Show
          Robert Joseph Evans added a comment - Refactored SchedulerNodeReport and SchedulerAppReport to not be a part of ScheduelrNode and SchedulerApp respectively.
          Hide
          Robert Joseph Evans added a comment -

          Canceling patch to do some more refactoring. Separating Scheduler*Report from Scheduler*, because, well Scheduler* is not really a report, so it is not as clean as I would like.

          Show
          Robert Joseph Evans added a comment - Canceling patch to do some more refactoring. Separating Scheduler*Report from Scheduler*, because, well Scheduler* is not really a report, so it is not as clean as I would like.
          Hide
          Robert Joseph Evans added a comment -

          Canceling patch to upmerge and rename the one class.

          Show
          Robert Joseph Evans added a comment - Canceling patch to upmerge and rename the one class.
          Hide
          Robert Joseph Evans added a comment -

          The reason the patch is smaller and simpler is because in the first patch I renamed SchedulerApp to SchedulerAppImpl. SchedulerApp is 20k so when the patch deletes and adds back in the same code it adds 40k to the size of the patch. If you want to change the names I can, it is not a big deal, but I don't think that Scheduler*/Scheduler*Impl really indicate the design pattern being used. I am not trying to separate out interface from implementation. There is just one implementation and no place in the code to allow another one to be inserted through configuration. I am just trying to expose a read only subset of the information about this data structure. After making the change to SchedulerNodeReport, I think it would be best to make the new interface SchedulerAppReport as well. This fits in with the naming conventions used elsewhere. I think I will upload a new patch with it changed to Report.

          Show
          Robert Joseph Evans added a comment - The reason the patch is smaller and simpler is because in the first patch I renamed SchedulerApp to SchedulerAppImpl. SchedulerApp is 20k so when the patch deletes and adds back in the same code it adds 40k to the size of the patch. If you want to change the names I can, it is not a big deal, but I don't think that Scheduler*/Scheduler*Impl really indicate the design pattern being used. I am not trying to separate out interface from implementation. There is just one implementation and no place in the code to allow another one to be inserted through configuration. I am just trying to expose a read only subset of the information about this data structure. After making the change to SchedulerNodeReport, I think it would be best to make the new interface SchedulerAppReport as well. This fits in with the naming conventions used elsewhere. I think I will upload a new patch with it changed to Report.
          Hide
          Arun C Murthy added a comment -

          Bobby, this seems a much simpler patch - nice!

          However, to keep things consistent in the RM codebase should we use SchedulerApp/SchedulerAppImpl and SchedulerNode/SchedulerNodeImpl?

          Show
          Arun C Murthy added a comment - Bobby, this seems a much simpler patch - nice! However, to keep things consistent in the RM codebase should we use SchedulerApp/SchedulerAppImpl and SchedulerNode/SchedulerNodeImpl?
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12497012/MR-3050.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/893//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/893//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497012/MR-3050.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/893//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/893//console This message is automatically generated.
          Hide
          Robert Joseph Evans added a comment -

          I refactored the code as was requested. The SchedulerNodeReport now has in the the usedResources and availableResources and the stand alone APIs were removed.

          I also changed SchedulerNodeReport into an interface because SchedulerNode was already the source of all of this data and it was simpler to have it implement that interface, then to try and extract the data to a new class just to return it.

          Show
          Robert Joseph Evans added a comment - I refactored the code as was requested. The SchedulerNodeReport now has in the the usedResources and availableResources and the stand alone APIs were removed. I also changed SchedulerNodeReport into an interface because SchedulerNode was already the source of all of this data and it was simpler to have it implement that interface, then to try and extract the data to a new class just to return it.
          Hide
          Robert Joseph Evans added a comment -

          Will do, and I will keep marking my jiras with MRV2 until I hear otherwise

          Thanks

          Show
          Robert Joseph Evans added a comment - Will do, and I will keep marking my jiras with MRV2 until I hear otherwise Thanks
          Hide
          Vinod Kumar Vavilapalli added a comment -

          I wasn't looking at this because of the missing Mrv2 tag

          If it is just hey Vinod take a look at this, then I am happy to use it as such

          I should've said, we* weren't looking, you know the vultures watching the PA list over and over
          We've been using mrv2 for all things marked for 0.23 that we'd definitely want to get pushed for the release.

          Regarding the patch, +1 for separating out the read-only interface. +1 for the names SchedulerAppInfo and SchedulerApp.

          Also, while you are at it, can you collapse the APIs Resource getUsedResource(NodeId nodeId) and Resource getAvailableResource(NodeId nodeId) into the single API SchedulerNodeReport getNodeReport(NodeId nodeId). SchedulerNodeReport can have all the information pertaining to a node.

          Show
          Vinod Kumar Vavilapalli added a comment - I wasn't looking at this because of the missing Mrv2 tag If it is just hey Vinod take a look at this, then I am happy to use it as such I should've said, we* weren't looking, you know the vultures watching the PA list over and over We've been using mrv2 for all things marked for 0.23 that we'd definitely want to get pushed for the release. Regarding the patch, +1 for separating out the read-only interface. +1 for the names SchedulerAppInfo and SchedulerApp. Also, while you are at it, can you collapse the APIs Resource getUsedResource(NodeId nodeId) and Resource getAvailableResource(NodeId nodeId) into the single API SchedulerNodeReport getNodeReport(NodeId nodeId) . SchedulerNodeReport can have all the information pertaining to a node.
          Hide
          Robert Joseph Evans added a comment -

          I will upmerge the patch. The reason for the split between SchedulerApp interface and SchedulerAppImpl is that SchedulerAppImpl seemed to expose way too many methods that should be specific to the scheduler. I wanted something that was read only that could not be used to accidentally manipulate the scheduler. I named them badly though. I will rename the interface to SchedulerAppInfo and the class back to SchedulerApp. I will look at SchedulerNode and see if it should take a SchedulerApp.

          Show
          Robert Joseph Evans added a comment - I will upmerge the patch. The reason for the split between SchedulerApp interface and SchedulerAppImpl is that SchedulerAppImpl seemed to expose way too many methods that should be specific to the scheduler. I wanted something that was read only that could not be used to accidentally manipulate the scheduler. I named them badly though. I will rename the interface to SchedulerAppInfo and the class back to SchedulerApp. I will look at SchedulerNode and see if it should take a SchedulerApp.
          Hide
          Mahadev konar added a comment -

          bobby, looks like the patch fails to apply. Can you please reupdate the patch. Its mostly the SchedulerApp changes.

          Qustions/Comments on the patch -

          • why did you have to seperate out SchedulerApp into interface and impl?
          • also shouldnt we be using the interface SchedulerApp in most of the cases if the above is needed?

          example - in SchedulerNode() methods we can just pass SchedulerNode and not SchedulerNodeImpl right?

          Show
          Mahadev konar added a comment - bobby, looks like the patch fails to apply. Can you please reupdate the patch. Its mostly the SchedulerApp changes. Qustions/Comments on the patch - why did you have to seperate out SchedulerApp into interface and impl? also shouldnt we be using the interface SchedulerApp in most of the cases if the above is needed? example - in SchedulerNode() methods we can just pass SchedulerNode and not SchedulerNodeImpl right?
          Hide
          Robert Joseph Evans added a comment -

          Sorry about that. I thought that because there is resourcemanager now and I did not touch the applicationmaster or the mr app client and the mrv2 code is in trunk that I did not need it. It almost seems that everything targeted to 0.23 or 0.24 and is a MAPREDCUE JIRA is mrv2. I am a bit confused about what mrv2 really means. If it is just hey Vinod take a look at this, then I am happy to use it as such .

          Show
          Robert Joseph Evans added a comment - Sorry about that. I thought that because there is resourcemanager now and I did not touch the applicationmaster or the mr app client and the mrv2 code is in trunk that I did not need it. It almost seems that everything targeted to 0.23 or 0.24 and is a MAPREDCUE JIRA is mrv2. I am a bit confused about what mrv2 really means. If it is just hey Vinod take a look at this, then I am happy to use it as such .
          Hide
          Vinod Kumar Vavilapalli added a comment -

          I wasn't looking at this because of the missing Mrv2 tag

          Reviewing the patch now, please gimme a day.

          Show
          Vinod Kumar Vavilapalli added a comment - I wasn't looking at this because of the missing Mrv2 tag Reviewing the patch now, please gimme a day.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12496697/MR-3050.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/870//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/870//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496697/MR-3050.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/870//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/870//console This message is automatically generated.
          Hide
          Robert Joseph Evans added a comment -

          Kicking it again to get Jenkins to work.

          Show
          Robert Joseph Evans added a comment - Kicking it again to get Jenkins to work.
          Hide
          Robert Joseph Evans added a comment -

          Uploading a new patch. I upmerged and changed getRootQueue to getRootQueueMetrics, because I did not like exposing the queue in that way.

          @Eric I looked more closely at your comments and I think you will want to use the API getSchedulerApp instead of creating a SchedulerApp yourself.

          Show
          Robert Joseph Evans added a comment - Uploading a new patch. I upmerged and changed getRootQueue to getRootQueueMetrics, because I did not like exposing the queue in that way. @Eric I looked more closely at your comments and I think you will want to use the API getSchedulerApp instead of creating a SchedulerApp yourself.
          Hide
          Robert Joseph Evans added a comment -

          Canceling the patch to address Eric's request.

          Show
          Robert Joseph Evans added a comment - Canceling the patch to address Eric's request.
          Hide
          Eric Payne added a comment -

          @Bobby,

          Would it be possible to change the SchedulerApp constructor to take a queue name instead of a Queue instance? I'm trying to call the SchedulerApp() constructor from the RMAppImpl class, and there's currently not a Scheduler agnostic way to get the Queue without knowing whether the scheduler is Fifo or Capacity.

          Show
          Eric Payne added a comment - @Bobby, Would it be possible to change the SchedulerApp constructor to take a queue name instead of a Queue instance? I'm trying to call the SchedulerApp() constructor from the RMAppImpl class, and there's currently not a Scheduler agnostic way to get the Queue without knowing whether the scheduler is Fifo or Capacity.
          Hide
          Eric Payne added a comment -

          +1 (non-binding)

          Thanks Bobby
          -Eric

          Show
          Eric Payne added a comment - +1 (non-binding) Thanks Bobby -Eric
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12496151/MR-3050.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/827//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/827//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496151/MR-3050.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/827//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/827//console This message is automatically generated.
          Hide
          Robert Joseph Evans added a comment -

          Incorporated review comments. Going to submit for real now.

          Show
          Robert Joseph Evans added a comment - Incorporated review comments. Going to submit for real now.
          Hide
          Robert Joseph Evans added a comment -

          SchedulerAppImpl.java: IIUC, getReservedContainers() is new. It seems to be very similar to getAllReservedContainers(). Do we need both, and if so, how are they different?

          getAllReservedContainers is synchronized properly where getReservedContainers is not . Also getReservedContainers is a public method exposed through the interface, so I will make just one method.

          YarnScheduler.java: getApplication() is @Private

          @Private is a thing that Hadoop invented and is not a standard java thing. This is just saying that even though the method is public according to java it is not intended to be used by other components. About the only thing it does is remove it's documentation from javadocs. I suppose LimitedPrivate("yarn") might be better, but I marked it @Private because Queue is @Private and I did not want to expose a @Private class from a @Public method.

          getRootQueue(): The implementation of this method returns different types from CapacityScheduler vs. FifoScheduler.

          The YarnScheduler interface returns a Queue. CapacityScheduler and FifoScheduler each return an implementation of Queue. What this means is that the following code will compile.

          YarnScheduler sched = ...;
          Queue q = sched.getRootQueue();
          if(sched instanceof CapcityScheduler) {
              CSQueue cq = ((CapacityScheduler)sched).getRootQueue();
          }
          
          Show
          Robert Joseph Evans added a comment - SchedulerAppImpl.java: IIUC, getReservedContainers() is new. It seems to be very similar to getAllReservedContainers(). Do we need both, and if so, how are they different? getAllReservedContainers is synchronized properly where getReservedContainers is not . Also getReservedContainers is a public method exposed through the interface, so I will make just one method. YarnScheduler.java: getApplication() is @Private @Private is a thing that Hadoop invented and is not a standard java thing. This is just saying that even though the method is public according to java it is not intended to be used by other components. About the only thing it does is remove it's documentation from javadocs. I suppose LimitedPrivate("yarn") might be better, but I marked it @Private because Queue is @Private and I did not want to expose a @Private class from a @Public method. getRootQueue(): The implementation of this method returns different types from CapacityScheduler vs. FifoScheduler. The YarnScheduler interface returns a Queue. CapacityScheduler and FifoScheduler each return an implementation of Queue. What this means is that the following code will compile. YarnScheduler sched = ...; Queue q = sched.getRootQueue(); if (sched instanceof CapcityScheduler) { CSQueue cq = ((CapacityScheduler)sched).getRootQueue(); }
          Hide
          Eric Payne added a comment -

          @Bobby

          I have a few questions:

          • SchedulerAppImpl.java: IIUC, getReservedContainers() is new. It seems to be very similar to getAllReservedContainers(). Do we need both, and if so, how are they different?
          • YarnScheduler.java: getApplication() is @Private, but the implementaion is public (CapacityScheduler, FifoScheduler). I thought Java complained when you do this? Why not just make the method interface @Public?. I guess the same question goes for getRootQueue().
          • getRootQueue(): The implementation of this method returns different types from CapacityScheduler vs. FifoScheduler. If I only have a reference to the YarnScheduler object (like in RMAppAttemptImpl), do I have to know whether it's a CapacityScheduler vs. FifoScheduler if I want to call getRootQueue()?
          Show
          Eric Payne added a comment - @Bobby I have a few questions: SchedulerAppImpl.java: IIUC, getReservedContainers() is new. It seems to be very similar to getAllReservedContainers() . Do we need both, and if so, how are they different? YarnScheduler.java: getApplication() is @Private, but the implementaion is public (CapacityScheduler, FifoScheduler). I thought Java complained when you do this? Why not just make the method interface @Public?. I guess the same question goes for getRootQueue() . getRootQueue() : The implementation of this method returns different types from CapacityScheduler vs. FifoScheduler. If I only have a reference to the YarnScheduler object (like in RMAppAttemptImpl), do I have to know whether it's a CapacityScheduler vs. FifoScheduler if I want to call getRootQueue() ?
          Hide
          Ravi Prakash added a comment -

          Thanks Bobby! Yes! I think it might be used in the future so can we please add it back in?

          I will also add in some comments to the AppScheduler to describe what it represents.

          Sweet! Thanks!

          Show
          Ravi Prakash added a comment - Thanks Bobby! Yes! I think it might be used in the future so can we please add it back in? I will also add in some comments to the AppScheduler to describe what it represents. Sweet! Thanks!
          Hide
          Robert Joseph Evans added a comment -

          I removed the queueComparator because it is private and not used by anything in the class. It is unrelated to this work, and I could add it back in if you thought it would be good.

          Changing the name to getSchedulerApp would probably be better too. I will also add in some comments to the AppScheduler to describe what it represents.

          Show
          Robert Joseph Evans added a comment - I removed the queueComparator because it is private and not used by anything in the class. It is unrelated to this work, and I could add it back in if you thought it would be good. Changing the name to getSchedulerApp would probably be better too. I will also add in some comments to the AppScheduler to describe what it represents.
          Hide
          Ravi Prakash added a comment -

          Hi Bobby,

          Looks great. Thanks for doing this My MR-2790 will be eternally grateful :-D

          nitpicks:

          1. Why did you have to remove queueComparator from ParentQueue? You haven't changed the signature of the constructor.
          2. YarnScheduler.java : the method name getApplication could be getSchedulerApp?
          Show
          Ravi Prakash added a comment - Hi Bobby, Looks great. Thanks for doing this My MR-2790 will be eternally grateful :-D nitpicks: Why did you have to remove queueComparator from ParentQueue? You haven't changed the signature of the constructor. YarnScheduler.java : the method name getApplication could be getSchedulerApp?
          Hide
          Robert Joseph Evans added a comment -

          This is the first draft of the patch. It is intended mostly for feedback on the APIs that I am adding to YarnScheduler.

            /**
             * Get the Scheduler app for a given app attempt Id.
             * @param appAttemptId the id of the application attempt
             * @return SchedulerApp for this given attempt.
             */
            @Private
            @Stable
            SchedulerApp getApplication(ApplicationAttemptId appAttemptId);
          
            /**
             * Get the root queue for the scheduler.
             * @return the root queue for the scheduler.
             */
            @Private
            @Evolving
            Queue getRootQueue();
          

          ShedulerApp is changed to SchedulerAppImpl and a new interface ScheduelerApp is created.

          @Evolving
          @LimitedPrivate("yarn")
          public interface SchedulerApp {
            public Collection<RMContainer> getLiveContainers();
            public Collection<RMContainer> getReservedContainers();
            public boolean isPending();
          }
          

          Any feedback on this is appreciated.

          Show
          Robert Joseph Evans added a comment - This is the first draft of the patch. It is intended mostly for feedback on the APIs that I am adding to YarnScheduler. /** * Get the Scheduler app for a given app attempt Id. * @param appAttemptId the id of the application attempt * @ return SchedulerApp for this given attempt. */ @Private @Stable SchedulerApp getApplication(ApplicationAttemptId appAttemptId); /** * Get the root queue for the scheduler. * @ return the root queue for the scheduler. */ @Private @Evolving Queue getRootQueue(); ShedulerApp is changed to SchedulerAppImpl and a new interface ScheduelerApp is created. @Evolving @LimitedPrivate( "yarn" ) public interface SchedulerApp { public Collection<RMContainer> getLiveContainers(); public Collection<RMContainer> getReservedContainers(); public boolean isPending(); } Any feedback on this is appreciated.
          Hide
          Robert Joseph Evans added a comment -

          It looks like that through the @Private APIs

          public SchedulerNodeReport getNodeReport(NodeId nodeId);
          Resource getUsedResource(NodeId nodeId);
          Resource getAvailableResource(NodeId nodeId);
          

          The Node information can be satisfied. So no need to add in new APIs to the Scheduler.

          Show
          Robert Joseph Evans added a comment - It looks like that through the @Private APIs public SchedulerNodeReport getNodeReport(NodeId nodeId); Resource getUsedResource(NodeId nodeId); Resource getAvailableResource(NodeId nodeId); The Node information can be satisfied. So no need to add in new APIs to the Scheduler.
          Hide
          Robert Joseph Evans added a comment -

          OK so to summarize the data that is needed is the following:

          • Total applications submitted since restart.
          • Total number of containers running.
          • Total memory(Resource) usage for running containers.
          • Total capacity memory(Resource) of the cluster.
          • Total memory(Resource) reserved in the cluster.
          • Given an ApplicationAttemptId get the
            • Containers for that attempt
            • Current resource utilization for that attempt
            • Current reserved resources for that attempt
          • Given a NodeId get the
            • Number of containers on the node
            • Memory (Resource) used
            • Memory (Resource) available

          @Ravi I don't think there is any way for the RM to get the no of maps and the number of reduces because that is Map/Reduce specific and you should be talking to the AppMaster/HistoryServer for that information.

          Show
          Robert Joseph Evans added a comment - OK so to summarize the data that is needed is the following: Total applications submitted since restart. Total number of containers running. Total memory(Resource) usage for running containers. Total capacity memory(Resource) of the cluster. Total memory(Resource) reserved in the cluster. Given an ApplicationAttemptId get the Containers for that attempt Current resource utilization for that attempt Current reserved resources for that attempt Given a NodeId get the Number of containers on the node Memory (Resource) used Memory (Resource) available @Ravi I don't think there is any way for the RM to get the no of maps and the number of reduces because that is Map/Reduce specific and you should be talking to the AppMaster/HistoryServer for that information.
          Hide
          Ravi Prakash added a comment -

          I need the number of containers, no. of maps, reduces and reservations too for MAPREDUCE-2790. This jira is VERY similar to 2789

          Show
          Ravi Prakash added a comment - I need the number of containers, no. of maps, reduces and reservations too for MAPREDUCE-2790 . This jira is VERY similar to 2789

            People

            • Assignee:
              Robert Joseph Evans
              Reporter:
              Robert Joseph Evans
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development