Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4719

Add a helper library to maintain node state and allows common queries

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: scheduler
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The scheduler could use a helper library to maintain node state and allowing matching/sorting queries. Several reasons for this:

      1. Today, a lot of the node state management is done separately in each scheduler. Having a single library will take us that much closer to reducing duplication among schedulers.
      2. Adding a filtering/matching API would simplify node labels and locality significantly.
      3. An API that returns a sorted list for a custom comparator would help YARN-1011 where we want to sort by allocation and utilization for continuous/asynchronous and opportunistic scheduling respectively.
      1. yarn-4719-1.patch
        46 kB
        Karthik Kambatla
      2. yarn-4719-2.patch
        49 kB
        Karthik Kambatla
      3. yarn-4719-3.patch
        56 kB
        Karthik Kambatla
      4. yarn-4719-4.patch
        61 kB
        Karthik Kambatla
      5. yarn-4719-5.patch
        62 kB
        Karthik Kambatla
      6. yarn-4719-6.patch
        62 kB
        Karthik Kambatla
      7. yarn-4719-7.patch
        63 kB
        Karthik Kambatla

        Activity

        Hide
        kasha Karthik Kambatla added a comment -

        First patch. It removes redundancy across schedulers. Kicking off Jenkins to see what all breaks.

        Show
        kasha Karthik Kambatla added a comment - First patch. It removes redundancy across schedulers. Kicking off Jenkins to see what all breaks.
        Hide
        kasha Karthik Kambatla added a comment -

        Ran Scheduler tests locally. Fixed most failures. Ran into a couple of failures that seem like test issues. Let us see what Jenkins thinks of the new patch.

        Show
        kasha Karthik Kambatla added a comment - Ran Scheduler tests locally. Fixed most failures. Ran into a couple of failures that seem like test issues. Let us see what Jenkins thinks of the new patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 5s trunk passed
        +1 compile 0m 28s trunk passed with JDK v1.8.0_72
        +1 compile 0m 28s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 36s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 1m 5s trunk passed
        +1 javadoc 0m 22s trunk passed with JDK v1.8.0_72
        +1 javadoc 0m 27s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 30s the patch passed
        +1 compile 0m 23s the patch passed with JDK v1.8.0_72
        +1 javac 0m 23s the patch passed
        +1 compile 0m 27s the patch passed with JDK v1.7.0_95
        +1 javac 0m 27s the patch passed
        -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 11 new + 285 unchanged - 4 fixed = 296 total (was 289)
        +1 mvnsite 0m 32s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        -1 findbugs 1m 16s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0)
        +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72
        +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95
        -1 unit 59m 18s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72.
        -1 unit 59m 30s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        135m 57s



        Reason Tests
        FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.addNode(SchedulerNode) invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:[line 70]
          Null pointer dereference of node in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:[line 139]
          Load of known null value in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:[line 139]
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) does not release lock on all exception paths At ClusterNodeTracker.java:on all exception paths At ClusterNodeTracker.java:[line 135]
          Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.removeNode(RMNode) Redundant null check at CapacityScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.removeNode(RMNode) Redundant null check at CapacityScheduler.java:[line 1486]
          Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.removeNode(RMNode) Redundant null check at FairScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.removeNode(RMNode) Redundant null check at FairScheduler.java:[line 903]
          Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:[line 930]
        JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestRMRestart
          hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
          hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
          hadoop.yarn.server.resourcemanager.TestRMHA
          hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
          hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
          hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
          hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
          hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
          hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
        JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
          org.apache.hadoop.yarn.server.resourcemanager.TestRM
          org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestRMRestart
          hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
          hadoop.yarn.server.resourcemanager.TestRMHA
          hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
          hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
          hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
          hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
          hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
          hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
        JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
          org.apache.hadoop.yarn.server.resourcemanager.TestRM
          org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService
          org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790385/yarn-4719-1.patch
        JIRA Issue YARN-4719
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 13382c48ea06 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 321a80c
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10659/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10659/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 5s trunk passed +1 compile 0m 28s trunk passed with JDK v1.8.0_72 +1 compile 0m 28s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 36s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 5s trunk passed +1 javadoc 0m 22s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 30s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_72 +1 javac 0m 23s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_95 +1 javac 0m 27s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 11 new + 285 unchanged - 4 fixed = 296 total (was 289) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 1m 16s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95 -1 unit 59m 18s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. -1 unit 59m 30s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 135m 57s Reason Tests FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager   org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.addNode(SchedulerNode) invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java: [line 70]   Null pointer dereference of node in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java: [line 139]   Load of known null value in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java: [line 139]   org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) does not release lock on all exception paths At ClusterNodeTracker.java:on all exception paths At ClusterNodeTracker.java: [line 135]   Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.removeNode(RMNode) Redundant null check at CapacityScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.removeNode(RMNode) Redundant null check at CapacityScheduler.java: [line 1486]   Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.removeNode(RMNode) Redundant null check at FairScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.removeNode(RMNode) Redundant null check at FairScheduler.java: [line 903]   Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java: [line 930] JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.security.TestAMRMTokens   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestRMHA   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   org.apache.hadoop.yarn.server.resourcemanager.TestRM   org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.security.TestAMRMTokens   hadoop.yarn.server.resourcemanager.TestRMHA   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   org.apache.hadoop.yarn.server.resourcemanager.TestRM   org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService   org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790385/yarn-4719-1.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 13382c48ea06 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 321a80c Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html unit https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10659/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10659/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 6m 33s trunk passed
        +1 compile 0m 25s trunk passed with JDK v1.8.0_72
        +1 compile 0m 28s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 34s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 4s trunk passed
        +1 javadoc 0m 21s trunk passed with JDK v1.8.0_72
        +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 29s the patch passed
        +1 compile 0m 23s the patch passed with JDK v1.8.0_72
        +1 javac 0m 23s the patch passed
        +1 compile 0m 27s the patch passed with JDK v1.7.0_95
        +1 javac 0m 27s the patch passed
        -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 12 new + 281 unchanged - 8 fixed = 293 total (was 289)
        +1 mvnsite 0m 32s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        -1 findbugs 1m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0)
        +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72
        +1 javadoc 0m 23s the patch passed with JDK v1.7.0_95
        -1 unit 71m 5s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72.
        -1 unit 76m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 36s Patch does not generate ASF License warnings.
        164m 24s



        Reason Tests
        FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.addNode(SchedulerNode) invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:[line 71]
          Null pointer dereference of node in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:[line 140]
          Load of known null value in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:[line 140]
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) does not release lock on all exception paths At ClusterNodeTracker.java:on all exception paths At ClusterNodeTracker.java:[line 136]
          Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:[line 930]
        JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
          hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
          hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
          hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790396/yarn-4719-2.patch
        JIRA Issue YARN-4719
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e958a90eb8f0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e3ac231
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10660/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10660/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 33s trunk passed +1 compile 0m 25s trunk passed with JDK v1.8.0_72 +1 compile 0m 28s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 34s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 4s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 29s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_72 +1 javac 0m 23s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_95 +1 javac 0m 27s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 12 new + 281 unchanged - 8 fixed = 293 total (was 289) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 1m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 23s the patch passed with JDK v1.7.0_95 -1 unit 71m 5s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. -1 unit 76m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 36s Patch does not generate ASF License warnings. 164m 24s Reason Tests FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager   org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.addNode(SchedulerNode) invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java: [line 71]   Null pointer dereference of node in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java: [line 140]   Load of known null value in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java: [line 140]   org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) does not release lock on all exception paths At ClusterNodeTracker.java:on all exception paths At ClusterNodeTracker.java: [line 136]   Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java: [line 930] JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790396/yarn-4719-2.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e958a90eb8f0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e3ac231 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html unit https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10660/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10660/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        asuresh Arun Suresh added a comment -

        Much needed patch Karthik Kambatla..

        Took a quick look, some comments

        1. You have an un-used import in the FairScheduler
        2. I see that you are triggering the update thread on nodeRemoval too. I understand this might generally be useful (since the node removal might change the node ordering), but given this is a refactoring patch, maybe address that separately ?
        3. May also be useful to expose functionality in the ClusterNodeTracker to give list of nodes in a rack, nodes that match a label expression etc. (This can possibly be another JIRA too)
        Show
        asuresh Arun Suresh added a comment - Much needed patch Karthik Kambatla .. Took a quick look, some comments You have an un-used import in the FairScheduler I see that you are triggering the update thread on nodeRemoval too. I understand this might generally be useful (since the node removal might change the node ordering), but given this is a refactoring patch, maybe address that separately ? May also be useful to expose functionality in the ClusterNodeTracker to give list of nodes in a rack, nodes that match a label expression etc. (This can possibly be another JIRA too)
        Hide
        kasha Karthik Kambatla added a comment -

        May also be useful to expose functionality in the ClusterNodeTracker to give list of nodes in a rack, nodes that match a label expression etc. (This can possibly be another JIRA too)

        Absolutely. I wanted to move all existing common functionality into this class in this JIRA, so we can add other helper functionality in the future.

        I see that you are triggering the update thread on nodeRemoval too. I understand this might generally be useful (since the node removal might change the node ordering), but given this is a refactoring patch, maybe address that separately ?

        removeNode does a triggerUpdate today too. I just moved it a little.

        Will fix the import and the test failures here in the next iteration.

        Show
        kasha Karthik Kambatla added a comment - May also be useful to expose functionality in the ClusterNodeTracker to give list of nodes in a rack, nodes that match a label expression etc. (This can possibly be another JIRA too) Absolutely. I wanted to move all existing common functionality into this class in this JIRA, so we can add other helper functionality in the future. I see that you are triggering the update thread on nodeRemoval too. I understand this might generally be useful (since the node removal might change the node ordering), but given this is a refactoring patch, maybe address that separately ? removeNode does a triggerUpdate today too. I just moved it a little. Will fix the import and the test failures here in the next iteration.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Hi Karthik Kambatla,

        Thanks for working on this patch, this is very useful.

        Took a very quick look at the patch, few comments:

        • For ClusterNodeTracker#nodes, can we use lock-free data structure to avoid copying the whole set?
        • We'd better not add addBlacklistedNodeIdsToList to ClusterNodeTracker since it calls application's logic, we should only include node related stuffs to ClusterNodeTracker.
        Show
        leftnoteasy Wangda Tan added a comment - Hi Karthik Kambatla , Thanks for working on this patch, this is very useful. Took a very quick look at the patch, few comments: For ClusterNodeTracker#nodes, can we use lock-free data structure to avoid copying the whole set? We'd better not add addBlacklistedNodeIdsToList to ClusterNodeTracker since it calls application's logic, we should only include node related stuffs to ClusterNodeTracker.
        Hide
        kasha Karthik Kambatla added a comment -

        For ClusterNodeTracker#nodes, can we use lock-free data structure to avoid copying the whole set?

        Not sure I understand the suggestion. Elaborate?

        We'd better not add addBlacklistedNodeIdsToList to ClusterNodeTracker since it calls application's logic, we should only include node related stuffs to ClusterNodeTracker.

        I feel any logic that has to iterate through all nodes should go through ClusterNodeTracker - that way, we don't run into cases where we access the list of nodes without a lock. Alternatively, we could get a list of nodeIDs from ClusterNodeTracker and then look up individual nodes. I am not particular about which approach, but also I don't quite see an issue with it being a part of ClusterNodeTracker. Any particular reason you think this doesn't belong here?

        Show
        kasha Karthik Kambatla added a comment - For ClusterNodeTracker#nodes, can we use lock-free data structure to avoid copying the whole set? Not sure I understand the suggestion. Elaborate? We'd better not add addBlacklistedNodeIdsToList to ClusterNodeTracker since it calls application's logic, we should only include node related stuffs to ClusterNodeTracker. I feel any logic that has to iterate through all nodes should go through ClusterNodeTracker - that way, we don't run into cases where we access the list of nodes without a lock. Alternatively, we could get a list of nodeIDs from ClusterNodeTracker and then look up individual nodes. I am not particular about which approach, but also I don't quite see an issue with it being a part of ClusterNodeTracker. Any particular reason you think this doesn't belong here?
        Hide
        kasha Karthik Kambatla added a comment -

        Updated patch should fix the test failures and findbugs warnings.

        Show
        kasha Karthik Kambatla added a comment - Updated patch should fix the test failures and findbugs warnings.
        Hide
        kasha Karthik Kambatla added a comment -

        Robert Kanter - this code touches the adaptive max allocation you have worked on. Mind taking a look to make sure I didn't screw up any of that.

        Show
        kasha Karthik Kambatla added a comment - Robert Kanter - this code touches the adaptive max allocation you have worked on. Mind taking a look to make sure I didn't screw up any of that.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Karthik Kambatla,

        Not sure I understand the suggestion. Elaborate?

        In ver.2 patch, getAllNodes uses shallowCopy, what I meant is instead of copying the entire HashMap, you can use ConcurrentMap instead.
        In ver.3 patch, you removed shallowCopy and returns HashMap.values(), if node removed while someone is iterating values(), the behavior is undefined. See: javadoc

        I feel any logic that has to iterate through all nodes should go through ClusterNodeTracker - that way, we don't run into cases where we access the list of nodes without a lock.

        As I commented above, we can use ConcurrentMap instead of locking ClusterNodeTracker. Do you need strong consistency for addBlacklistedNodeIdsToList? (Because node list could be updated while we updating blacklistedNodes.

        Any particular reason you think this doesn't belong here?

        I would prefer to keep cleaner responsibility of ClusterNodeTracker, if we adds application logic here, we could add any logic related to SchedulerNode to this class as well. This refactoring patch is majorly for code clean up to me, I think it's better to keep it clean from the beginning.

        Show
        leftnoteasy Wangda Tan added a comment - Karthik Kambatla , Not sure I understand the suggestion. Elaborate? In ver.2 patch, getAllNodes uses shallowCopy, what I meant is instead of copying the entire HashMap, you can use ConcurrentMap instead. In ver.3 patch, you removed shallowCopy and returns HashMap.values(), if node removed while someone is iterating values(), the behavior is undefined. See: javadoc I feel any logic that has to iterate through all nodes should go through ClusterNodeTracker - that way, we don't run into cases where we access the list of nodes without a lock. As I commented above, we can use ConcurrentMap instead of locking ClusterNodeTracker. Do you need strong consistency for addBlacklistedNodeIdsToList? (Because node list could be updated while we updating blacklistedNodes. Any particular reason you think this doesn't belong here? I would prefer to keep cleaner responsibility of ClusterNodeTracker, if we adds application logic here, we could add any logic related to SchedulerNode to this class as well. This refactoring patch is majorly for code clean up to me, I think it's better to keep it clean from the beginning.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
        +1 mvninstall 7m 59s trunk passed
        +1 compile 0m 36s trunk passed with JDK v1.8.0_72
        +1 compile 0m 33s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 24s trunk passed
        +1 mvnsite 0m 40s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 17s trunk passed
        +1 javadoc 0m 25s trunk passed with JDK v1.8.0_72
        +1 javadoc 0m 28s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 35s the patch passed
        +1 compile 0m 31s the patch passed with JDK v1.8.0_72
        +1 javac 0m 31s the patch passed
        +1 compile 0m 29s the patch passed with JDK v1.7.0_95
        -1 javac 3m 59s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2)
        +1 javac 0m 29s the patch passed
        -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 9 new + 278 unchanged - 8 fixed = 287 total (was 286)
        +1 mvnsite 0m 36s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 28s the patch passed
        +1 javadoc 0m 24s the patch passed with JDK v1.8.0_72
        +1 javadoc 0m 28s the patch passed with JDK v1.7.0_95
        -1 unit 68m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72.
        -1 unit 68m 47s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        156m 28s



        Reason Tests
        JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790991/yarn-4719-3.patch
        JIRA Issue YARN-4719
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux d84e64254cca 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 67880cc
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10687/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10687/console
        Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 7m 59s trunk passed +1 compile 0m 36s trunk passed with JDK v1.8.0_72 +1 compile 0m 33s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 24s trunk passed +1 mvnsite 0m 40s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 17s trunk passed +1 javadoc 0m 25s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 28s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 35s the patch passed +1 compile 0m 31s the patch passed with JDK v1.8.0_72 +1 javac 0m 31s the patch passed +1 compile 0m 29s the patch passed with JDK v1.7.0_95 -1 javac 3m 59s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2) +1 javac 0m 29s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 9 new + 278 unchanged - 8 fixed = 287 total (was 286) +1 mvnsite 0m 36s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 28s the patch passed +1 javadoc 0m 24s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 28s the patch passed with JDK v1.7.0_95 -1 unit 68m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. -1 unit 68m 47s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 156m 28s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790991/yarn-4719-3.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d84e64254cca 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 67880cc Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10687/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10687/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kasha Karthik Kambatla added a comment -

        Wangda Tan - thanks for chiming in, you make some valid points.

        Since we are building a library for node tracking, I would like for us to restrict access to the map/set of nodes tracked only through addNode and removeNode so total_cluster_resources, total_inflated_cluster_resources (for YARN-1011), max_cluster_resources are not affected by other scheduler code. Do you think this is a reasonable goal? At least, as long as it doesn't hurt performance?

        If yes, we should decide on how to handle cases where the scheduler code needs to iterate through the nodes: (1) we could provide a snapshot copy of the map/set of nodes/nodeIds, or (2) provide a way to do the same with the right locks by adding additional methods or an abstraction (similar to lambdas) that applies to multiple methods.

        Thoughts?

        PS: By the way, thanks for pointing out the javadoc for values(). I will clean that up based on the discussion output here.

        Show
        kasha Karthik Kambatla added a comment - Wangda Tan - thanks for chiming in, you make some valid points. Since we are building a library for node tracking, I would like for us to restrict access to the map/set of nodes tracked only through addNode and removeNode so total_cluster_resources, total_inflated_cluster_resources (for YARN-1011 ), max_cluster_resources are not affected by other scheduler code. Do you think this is a reasonable goal? At least, as long as it doesn't hurt performance? If yes, we should decide on how to handle cases where the scheduler code needs to iterate through the nodes: (1) we could provide a snapshot copy of the map/set of nodes/nodeIds, or (2) provide a way to do the same with the right locks by adding additional methods or an abstraction (similar to lambdas) that applies to multiple methods. Thoughts? PS: By the way, thanks for pointing out the javadoc for values(). I will clean that up based on the discussion output here.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Hi Karthik Kambatla,

        through addNode and removeNode so total_cluster_resources, total_inflated_cluster_resources (for YARN-1011), max_cluster_resources are not affected by other scheduler code.

        I may not understand about this, could you elaborate?

        For handle scheduler code to iterate nodes, we could either:

        1. Use concurrent map to avoid locking, and code will not break. Drawback: we need to handle stale data.
        2. Expose lock to external caller, so scheduler can get readlock of ClusterNodeTracker and do iteration. Drawback: iteration nodes and allocating containers could lock ClusterNodeTracker for long time.
        3. Assume synchronize lock of scheduler will be acquired when make changes to ClusterNodeTracker (like addNode, removeNode, etc.), and also when iterating nodes. We don't need extra lock of returned node collections. Drawback: this hides locks to external caller behaviors, and in the future scheduler could remove synchronized lock to get better performance.

        I would suggest to look at if #1 is doable (handle stale data and assumes eventually consistency). #1 should have best performance and flexible to future scheduler changes.

        Show
        leftnoteasy Wangda Tan added a comment - Hi Karthik Kambatla , through addNode and removeNode so total_cluster_resources, total_inflated_cluster_resources (for YARN-1011 ), max_cluster_resources are not affected by other scheduler code. I may not understand about this, could you elaborate? For handle scheduler code to iterate nodes, we could either: Use concurrent map to avoid locking, and code will not break. Drawback: we need to handle stale data. Expose lock to external caller, so scheduler can get readlock of ClusterNodeTracker and do iteration. Drawback: iteration nodes and allocating containers could lock ClusterNodeTracker for long time. Assume synchronize lock of scheduler will be acquired when make changes to ClusterNodeTracker (like addNode, removeNode, etc.), and also when iterating nodes. We don't need extra lock of returned node collections. Drawback: this hides locks to external caller behaviors, and in the future scheduler could remove synchronized lock to get better performance. I would suggest to look at if #1 is doable (handle stale data and assumes eventually consistency). #1 should have best performance and flexible to future scheduler changes.
        Hide
        kasha Karthik Kambatla added a comment -

        I may not understand about this, could you elaborate?

        When we add or remove a node, a few things are updated:

        1. The map/set holding these nodes
        2. Total cluster capacity
        3. Total inflated cluster capacity (that takes allocated but not utilized resources into account)
        4. Largest container that could potentially be allocated

        As long as it doesn't interfere with performance, I would like to keep all of this information consistent. That is if one thread has added a node, another thread shouldn't see a stale value for cluster capacity. Ensuring this consistency requires us to hold write locks when adding or removing a node, and read locks when accessing any of this information. Given that the current code has been ensuring consistency through a lock on the scheduler (at least FairScheduler), I doubt adding these read/write locks would lead to performance issues. I haven't done any benchmarking though.

        Do you agree with the approach of consistency-over-performance to begin with? If yes, we need the locks. And, making the HashMap a ConcurrentHashMap wouldn't buy us much.

        Coming to iterating over the nodes, I agree with your concern that we might proliferate ClusterNodeTracker with methods like addBlacklistedNodeIdsToList. And, understand your point that using a ConcurrentHashMap would allow us to iterate freely on the snapshot data without exposing the internals. That said, given that there are likely only a handful of cases where iterating through all the nodes is necessary and one of the goals for this library to help with preemption, node-labeling, and over-subscription, what do you think of the following construct:

          public List<NodeId> filter(NodeFilter nodeFilter) {
            List<NodeId> nodeIds = new ArrayList<>();
            readLock.lock();
            try {
              for (N node : nodes.values()) {
                if (nodeFilter.accept(node)) {
                  nodeIds.add(node.getNodeID());
                }
              }
            } finally {
              readLock.unlock();
            }
            return nodeIds;
          }
        

        addBlackListedNodeIdsToList can implement a trivial NodeFilter.accept as SchedulerAppUtils.isBlacklisted(app, node, LOG). We have used this approach to great effect with PathFilter.

        Thoughts?

        Show
        kasha Karthik Kambatla added a comment - I may not understand about this, could you elaborate? When we add or remove a node, a few things are updated: The map/set holding these nodes Total cluster capacity Total inflated cluster capacity (that takes allocated but not utilized resources into account) Largest container that could potentially be allocated As long as it doesn't interfere with performance, I would like to keep all of this information consistent. That is if one thread has added a node, another thread shouldn't see a stale value for cluster capacity. Ensuring this consistency requires us to hold write locks when adding or removing a node, and read locks when accessing any of this information. Given that the current code has been ensuring consistency through a lock on the scheduler (at least FairScheduler), I doubt adding these read/write locks would lead to performance issues. I haven't done any benchmarking though. Do you agree with the approach of consistency-over-performance to begin with? If yes, we need the locks. And, making the HashMap a ConcurrentHashMap wouldn't buy us much. Coming to iterating over the nodes, I agree with your concern that we might proliferate ClusterNodeTracker with methods like addBlacklistedNodeIdsToList. And, understand your point that using a ConcurrentHashMap would allow us to iterate freely on the snapshot data without exposing the internals. That said, given that there are likely only a handful of cases where iterating through all the nodes is necessary and one of the goals for this library to help with preemption, node-labeling, and over-subscription, what do you think of the following construct: public List<NodeId> filter(NodeFilter nodeFilter) { List<NodeId> nodeIds = new ArrayList<>(); readLock.lock(); try { for (N node : nodes.values()) { if (nodeFilter.accept(node)) { nodeIds.add(node.getNodeID()); } } } finally { readLock.unlock(); } return nodeIds; } addBlackListedNodeIdsToList can implement a trivial NodeFilter.accept as SchedulerAppUtils.isBlacklisted(app, node, LOG) . We have used this approach to great effect with PathFilter. Thoughts?
        Hide
        kasha Karthik Kambatla added a comment -

        Oh, and when the NodeFilter is null or an "IdentityFilter" (that always returns true), we could choose to skip iteration and just populate the list with the keySet(). That way, we don't need a separate method to return all nodes.

        Show
        kasha Karthik Kambatla added a comment - Oh, and when the NodeFilter is null or an "IdentityFilter" (that always returns true), we could choose to skip iteration and just populate the list with the keySet(). That way, we don't need a separate method to return all nodes.
        Hide
        kasha Karthik Kambatla added a comment -

        Patch that uses NodeFilter to keep non-node-tracking logic out of ClusterNodeTracker. Wangda Tan - what do you think of this approach?

        Show
        kasha Karthik Kambatla added a comment - Patch that uses NodeFilter to keep non-node-tracking logic out of ClusterNodeTracker. Wangda Tan - what do you think of this approach?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 7s YARN-4719 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792015/yarn-4719-4.patch
        JIRA Issue YARN-4719
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10721/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 7s YARN-4719 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792015/yarn-4719-4.patch JIRA Issue YARN-4719 Console output https://builds.apache.org/job/PreCommit-YARN-Build/10721/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 4s YARN-4719 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792018/yarn-4719-4.patch
        JIRA Issue YARN-4719
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10722/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 4s YARN-4719 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792018/yarn-4719-4.patch JIRA Issue YARN-4719 Console output https://builds.apache.org/job/PreCommit-YARN-Build/10722/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
        +1 mvninstall 7m 26s trunk passed
        +1 compile 0m 31s trunk passed with JDK v1.8.0_74
        +1 compile 0m 31s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 36s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 9s trunk passed
        +1 javadoc 0m 23s trunk passed with JDK v1.8.0_74
        +1 javadoc 0m 28s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 31s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.8.0_74
        +1 javac 0m 26s the patch passed
        +1 compile 0m 28s the patch passed with JDK v1.7.0_95
        -1 javac 3m 39s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2)
        +1 javac 0m 28s the patch passed
        -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 9 new + 304 unchanged - 8 fixed = 313 total (was 312)
        +1 mvnsite 0m 33s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 19s the patch passed
        +1 javadoc 0m 20s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 25s the patch passed with JDK v1.7.0_95
        -1 unit 70m 35s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
        -1 unit 71m 56s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        160m 31s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792021/yarn-4719-4.patch
        JIRA Issue YARN-4719
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b84b1396099e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / f86850b
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10723/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10723/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 7m 26s trunk passed +1 compile 0m 31s trunk passed with JDK v1.8.0_74 +1 compile 0m 31s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 36s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 9s trunk passed +1 javadoc 0m 23s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 28s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 31s the patch passed +1 compile 0m 26s the patch passed with JDK v1.8.0_74 +1 javac 0m 26s the patch passed +1 compile 0m 28s the patch passed with JDK v1.7.0_95 -1 javac 3m 39s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2) +1 javac 0m 28s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 9 new + 304 unchanged - 8 fixed = 313 total (was 312) +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 19s the patch passed +1 javadoc 0m 20s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 25s the patch passed with JDK v1.7.0_95 -1 unit 70m 35s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. -1 unit 71m 56s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 160m 31s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792021/yarn-4719-4.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b84b1396099e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f86850b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10723/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10723/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        kasha Karthik Kambatla added a comment -

        v5 patch addresses javac and checkstyle warnings.

        Show
        kasha Karthik Kambatla added a comment - v5 patch addresses javac and checkstyle warnings.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
        +1 mvninstall 7m 0s trunk passed
        +1 compile 0m 28s trunk passed with JDK v1.8.0_74
        +1 compile 0m 30s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 34s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 7s trunk passed
        +1 javadoc 0m 21s trunk passed with JDK v1.8.0_74
        +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 31s the patch passed
        +1 compile 0m 27s the patch passed with JDK v1.8.0_74
        +1 javac 0m 27s the patch passed
        +1 compile 0m 28s the patch passed with JDK v1.7.0_95
        -1 javac 3m 32s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2)
        +1 javac 0m 28s the patch passed
        -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312)
        +1 mvnsite 0m 32s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 17s the patch passed
        +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95
        -1 unit 67m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
        -1 unit 69m 9s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        153m 33s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792148/yarn-4719-5.patch
        JIRA Issue YARN-4719
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 50a72bd69b5e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 55f73a1
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10730/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10730/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 7m 0s trunk passed +1 compile 0m 28s trunk passed with JDK v1.8.0_74 +1 compile 0m 30s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 34s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 7s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 31s the patch passed +1 compile 0m 27s the patch passed with JDK v1.8.0_74 +1 javac 0m 27s the patch passed +1 compile 0m 28s the patch passed with JDK v1.7.0_95 -1 javac 3m 32s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2) +1 javac 0m 28s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 17s the patch passed +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95 -1 unit 67m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. -1 unit 69m 9s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 153m 33s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792148/yarn-4719-5.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 50a72bd69b5e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 55f73a1 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10730/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10730/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        asuresh Arun Suresh added a comment -

        Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ?
        I can forsee many usecases of having pluggable implementation For eg. an in-memory / external Db that can handle filtering / matching of nodes more efficiently.

        Show
        asuresh Arun Suresh added a comment - Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ? I can forsee many usecases of having pluggable implementation For eg. an in-memory / external Db that can handle filtering / matching of nodes more efficiently.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Looks good to me in general, +1

        One nit: CS#reintializeQueues:

            root.reinitialize(newRoot, getClusterResource());
            updatePlacementRules();
        
            // Re-calculate headroom for active applications
            Resource clusterResource = nodeTracker.getClusterCapacity();
        

        Should use getClusterResource instead of nodeTracker.getClusterCapacity.

        Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ?

        I would suggest keep it as-is, we can refactor it in the future when we have to. Modifying interface needs some additional efforts, we should avoid them now.

        Show
        leftnoteasy Wangda Tan added a comment - Looks good to me in general, +1 One nit: CS#reintializeQueues: root.reinitialize(newRoot, getClusterResource()); updatePlacementRules(); // Re-calculate headroom for active applications Resource clusterResource = nodeTracker.getClusterCapacity(); Should use getClusterResource instead of nodeTracker.getClusterCapacity. Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ? I would suggest keep it as-is, we can refactor it in the future when we have to. Modifying interface needs some additional efforts, we should avoid them now.
        Hide
        kasha Karthik Kambatla added a comment -

        Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ?

        Good idea, but as Wangda says, we can do this lazily. The longer we wait, we probably also will have a good understanding of what the interface should be like.

        Updating patch to fix the nit shortly.

        Show
        kasha Karthik Kambatla added a comment - Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ? Good idea, but as Wangda says, we can do this lazily. The longer we wait, we probably also will have a good understanding of what the interface should be like. Updating patch to fix the nit shortly.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 17m 44s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
        +1 mvninstall 7m 1s trunk passed
        +1 compile 0m 29s trunk passed with JDK v1.8.0_74
        +1 compile 0m 29s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 35s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 6s trunk passed
        +1 javadoc 0m 21s trunk passed with JDK v1.8.0_74
        +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 31s the patch passed
        +1 compile 0m 27s the patch passed with JDK v1.8.0_74
        +1 javac 0m 27s the patch passed
        +1 compile 0m 27s the patch passed with JDK v1.7.0_95
        -1 javac 3m 32s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2)
        +1 javac 0m 26s the patch passed
        -1 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312)
        +1 mvnsite 0m 32s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 16s the patch passed
        +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95
        -1 unit 72m 30s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
        -1 unit 72m 34s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        179m 43s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792911/yarn-4719-6.patch
        JIRA Issue YARN-4719
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 215055afe2a1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 658ee95
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10766/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10766/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 17m 44s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 7m 1s trunk passed +1 compile 0m 29s trunk passed with JDK v1.8.0_74 +1 compile 0m 29s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 35s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 6s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 31s the patch passed +1 compile 0m 27s the patch passed with JDK v1.8.0_74 +1 javac 0m 27s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_95 -1 javac 3m 32s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2) +1 javac 0m 26s the patch passed -1 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 16s the patch passed +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95 -1 unit 72m 30s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. -1 unit 72m 34s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 179m 43s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792911/yarn-4719-6.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 215055afe2a1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 658ee95 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10766/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10766/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        kasha Karthik Kambatla added a comment -

        Patch to fix the javac warning. The checkstyle warnings are innocuous.

        Show
        kasha Karthik Kambatla added a comment - Patch to fix the javac warning. The checkstyle warnings are innocuous.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
        +1 mvninstall 6m 30s trunk passed
        +1 compile 0m 25s trunk passed with JDK v1.8.0_74
        +1 compile 0m 28s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 34s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 3s trunk passed
        +1 javadoc 0m 22s trunk passed with JDK v1.8.0_74
        +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 31s the patch passed
        +1 compile 0m 23s the patch passed with JDK v1.8.0_74
        +1 javac 0m 23s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.7.0_95
        +1 javac 0m 26s the patch passed
        -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312)
        +1 mvnsite 0m 31s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 15s the patch passed
        +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 23s the patch passed with JDK v1.7.0_95
        -1 unit 70m 38s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
        -1 unit 72m 7s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        159m 4s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793167/yarn-4719-7.patch
        JIRA Issue YARN-4719
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 84c4362d6820 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 658ee95
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10769/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10769/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 6m 30s trunk passed +1 compile 0m 25s trunk passed with JDK v1.8.0_74 +1 compile 0m 28s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 34s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 3s trunk passed +1 javadoc 0m 22s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 31s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_74 +1 javac 0m 23s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_95 +1 javac 0m 26s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312) +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 15s the patch passed +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 23s the patch passed with JDK v1.7.0_95 -1 unit 70m 38s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. -1 unit 72m 7s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 159m 4s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793167/yarn-4719-7.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 84c4362d6820 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 658ee95 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10769/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10769/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        kasha Karthik Kambatla added a comment -

        The checkstyle warnings are innocuous, and the test failures are known. Arun Suresh, Wangda Tan - can one of you bless the latest patch?

        Show
        kasha Karthik Kambatla added a comment - The checkstyle warnings are innocuous, and the test failures are known. Arun Suresh , Wangda Tan - can one of you bless the latest patch?
        Hide
        leftnoteasy Wangda Tan added a comment -

        +1, thanks Karthik Kambatla.

        Show
        leftnoteasy Wangda Tan added a comment - +1, thanks Karthik Kambatla .
        Hide
        kasha Karthik Kambatla added a comment -

        Thanks a bunch for the reviews, Arun Suresh and Wangda Tan. Excited to get this in.

        Just committed it to trunk and branch-2.

        Show
        kasha Karthik Kambatla added a comment - Thanks a bunch for the reviews, Arun Suresh and Wangda Tan . Excited to get this in. Just committed it to trunk and branch-2.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9460 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9460/)
        YARN-4719. Add a helper library to maintain node state and allows common (kasha: rev 20d389ce61eaacb5ddfb329015f50e96ad894f8d)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/NodeFilter.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestReservations.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ClusterNodeTracker.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9460 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9460/ ) YARN-4719 . Add a helper library to maintain node state and allows common (kasha: rev 20d389ce61eaacb5ddfb329015f50e96ad894f8d) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/NodeFilter.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestReservations.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ClusterNodeTracker.java

          People

          • Assignee:
            kasha Karthik Kambatla
            Reporter:
            kasha Karthik Kambatla
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development