Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4719

Add a helper library to maintain node state and allows common queries

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: scheduler
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The scheduler could use a helper library to maintain node state and allowing matching/sorting queries. Several reasons for this:

      1. Today, a lot of the node state management is done separately in each scheduler. Having a single library will take us that much closer to reducing duplication among schedulers.
      2. Adding a filtering/matching API would simplify node labels and locality significantly.
      3. An API that returns a sorted list for a custom comparator would help YARN-1011 where we want to sort by allocation and utilization for continuous/asynchronous and opportunistic scheduling respectively.
      1. yarn-4719-7.patch
        63 kB
        Karthik Kambatla
      2. yarn-4719-6.patch
        62 kB
        Karthik Kambatla
      3. yarn-4719-5.patch
        62 kB
        Karthik Kambatla
      4. yarn-4719-4.patch
        61 kB
        Karthik Kambatla
      5. yarn-4719-3.patch
        56 kB
        Karthik Kambatla
      6. yarn-4719-2.patch
        49 kB
        Karthik Kambatla
      7. yarn-4719-1.patch
        46 kB
        Karthik Kambatla

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9460 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9460/)
          YARN-4719. Add a helper library to maintain node state and allows common (kasha: rev 20d389ce61eaacb5ddfb329015f50e96ad894f8d)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/NodeFilter.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestReservations.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ClusterNodeTracker.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9460 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9460/ ) YARN-4719 . Add a helper library to maintain node state and allows common (kasha: rev 20d389ce61eaacb5ddfb329015f50e96ad894f8d) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/NodeFilter.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestReservations.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ClusterNodeTracker.java
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks a bunch for the reviews, Arun Suresh and Wangda Tan. Excited to get this in.

          Just committed it to trunk and branch-2.

          Show
          kasha Karthik Kambatla added a comment - Thanks a bunch for the reviews, Arun Suresh and Wangda Tan . Excited to get this in. Just committed it to trunk and branch-2.
          Hide
          leftnoteasy Wangda Tan added a comment -

          +1, thanks Karthik Kambatla.

          Show
          leftnoteasy Wangda Tan added a comment - +1, thanks Karthik Kambatla .
          Hide
          kasha Karthik Kambatla added a comment -

          The checkstyle warnings are innocuous, and the test failures are known. Arun Suresh, Wangda Tan - can one of you bless the latest patch?

          Show
          kasha Karthik Kambatla added a comment - The checkstyle warnings are innocuous, and the test failures are known. Arun Suresh , Wangda Tan - can one of you bless the latest patch?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          +1 mvninstall 6m 30s trunk passed
          +1 compile 0m 25s trunk passed with JDK v1.8.0_74
          +1 compile 0m 28s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 0m 34s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 3s trunk passed
          +1 javadoc 0m 22s trunk passed with JDK v1.8.0_74
          +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 31s the patch passed
          +1 compile 0m 23s the patch passed with JDK v1.8.0_74
          +1 javac 0m 23s the patch passed
          +1 compile 0m 26s the patch passed with JDK v1.7.0_95
          +1 javac 0m 26s the patch passed
          -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312)
          +1 mvnsite 0m 31s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 15s the patch passed
          +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74
          +1 javadoc 0m 23s the patch passed with JDK v1.7.0_95
          -1 unit 70m 38s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
          -1 unit 72m 7s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          159m 4s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793167/yarn-4719-7.patch
          JIRA Issue YARN-4719
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 84c4362d6820 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 658ee95
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10769/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10769/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 6m 30s trunk passed +1 compile 0m 25s trunk passed with JDK v1.8.0_74 +1 compile 0m 28s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 34s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 3s trunk passed +1 javadoc 0m 22s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 31s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_74 +1 javac 0m 23s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_95 +1 javac 0m 26s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312) +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 15s the patch passed +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 23s the patch passed with JDK v1.7.0_95 -1 unit 70m 38s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. -1 unit 72m 7s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 159m 4s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793167/yarn-4719-7.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 84c4362d6820 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 658ee95 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10769/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10769/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10769/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          Patch to fix the javac warning. The checkstyle warnings are innocuous.

          Show
          kasha Karthik Kambatla added a comment - Patch to fix the javac warning. The checkstyle warnings are innocuous.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 17m 44s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          +1 mvninstall 7m 1s trunk passed
          +1 compile 0m 29s trunk passed with JDK v1.8.0_74
          +1 compile 0m 29s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 0m 35s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 6s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_74
          +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 31s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.8.0_74
          +1 javac 0m 27s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.7.0_95
          -1 javac 3m 32s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2)
          +1 javac 0m 26s the patch passed
          -1 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312)
          +1 mvnsite 0m 32s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 16s the patch passed
          +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74
          +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95
          -1 unit 72m 30s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
          -1 unit 72m 34s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          179m 43s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792911/yarn-4719-6.patch
          JIRA Issue YARN-4719
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 215055afe2a1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 658ee95
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10766/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10766/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 17m 44s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 7m 1s trunk passed +1 compile 0m 29s trunk passed with JDK v1.8.0_74 +1 compile 0m 29s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 35s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 6s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 31s the patch passed +1 compile 0m 27s the patch passed with JDK v1.8.0_74 +1 javac 0m 27s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_95 -1 javac 3m 32s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2) +1 javac 0m 26s the patch passed -1 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 16s the patch passed +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95 -1 unit 72m 30s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. -1 unit 72m 34s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 179m 43s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792911/yarn-4719-6.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 215055afe2a1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 658ee95 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10766/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10766/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10766/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ?

          Good idea, but as Wangda says, we can do this lazily. The longer we wait, we probably also will have a good understanding of what the interface should be like.

          Updating patch to fix the nit shortly.

          Show
          kasha Karthik Kambatla added a comment - Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ? Good idea, but as Wangda says, we can do this lazily. The longer we wait, we probably also will have a good understanding of what the interface should be like. Updating patch to fix the nit shortly.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Looks good to me in general, +1

          One nit: CS#reintializeQueues:

              root.reinitialize(newRoot, getClusterResource());
              updatePlacementRules();
          
              // Re-calculate headroom for active applications
              Resource clusterResource = nodeTracker.getClusterCapacity();
          

          Should use getClusterResource instead of nodeTracker.getClusterCapacity.

          Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ?

          I would suggest keep it as-is, we can refactor it in the future when we have to. Modifying interface needs some additional efforts, we should avoid them now.

          Show
          leftnoteasy Wangda Tan added a comment - Looks good to me in general, +1 One nit: CS#reintializeQueues: root.reinitialize(newRoot, getClusterResource()); updatePlacementRules(); // Re-calculate headroom for active applications Resource clusterResource = nodeTracker.getClusterCapacity(); Should use getClusterResource instead of nodeTracker.getClusterCapacity. Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ? I would suggest keep it as-is, we can refactor it in the future when we have to. Modifying interface needs some additional efforts, we should avoid them now.
          Hide
          asuresh Arun Suresh added a comment -

          Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ?
          I can forsee many usecases of having pluggable implementation For eg. an in-memory / external Db that can handle filtering / matching of nodes more efficiently.

          Show
          asuresh Arun Suresh added a comment - Can we make ClusterNodeTracker an interface and make the current one in latest patch a Default Implementation ? I can forsee many usecases of having pluggable implementation For eg. an in-memory / external Db that can handle filtering / matching of nodes more efficiently.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          +1 mvninstall 7m 0s trunk passed
          +1 compile 0m 28s trunk passed with JDK v1.8.0_74
          +1 compile 0m 30s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 0m 34s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 7s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_74
          +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 31s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.8.0_74
          +1 javac 0m 27s the patch passed
          +1 compile 0m 28s the patch passed with JDK v1.7.0_95
          -1 javac 3m 32s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2)
          +1 javac 0m 28s the patch passed
          -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312)
          +1 mvnsite 0m 32s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 17s the patch passed
          +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74
          +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95
          -1 unit 67m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
          -1 unit 69m 9s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          153m 33s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792148/yarn-4719-5.patch
          JIRA Issue YARN-4719
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 50a72bd69b5e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 55f73a1
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10730/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10730/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 7m 0s trunk passed +1 compile 0m 28s trunk passed with JDK v1.8.0_74 +1 compile 0m 30s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 34s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 7s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 31s the patch passed +1 compile 0m 27s the patch passed with JDK v1.8.0_74 +1 javac 0m 27s the patch passed +1 compile 0m 28s the patch passed with JDK v1.7.0_95 -1 javac 3m 32s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2) +1 javac 0m 28s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 303 unchanged - 9 fixed = 305 total (was 312) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 17s the patch passed +1 javadoc 0m 19s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95 -1 unit 67m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. -1 unit 69m 9s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 153m 33s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792148/yarn-4719-5.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 50a72bd69b5e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 55f73a1 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10730/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10730/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10730/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          v5 patch addresses javac and checkstyle warnings.

          Show
          kasha Karthik Kambatla added a comment - v5 patch addresses javac and checkstyle warnings.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          +1 mvninstall 7m 26s trunk passed
          +1 compile 0m 31s trunk passed with JDK v1.8.0_74
          +1 compile 0m 31s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 0m 36s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 9s trunk passed
          +1 javadoc 0m 23s trunk passed with JDK v1.8.0_74
          +1 javadoc 0m 28s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 31s the patch passed
          +1 compile 0m 26s the patch passed with JDK v1.8.0_74
          +1 javac 0m 26s the patch passed
          +1 compile 0m 28s the patch passed with JDK v1.7.0_95
          -1 javac 3m 39s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2)
          +1 javac 0m 28s the patch passed
          -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 9 new + 304 unchanged - 8 fixed = 313 total (was 312)
          +1 mvnsite 0m 33s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 19s the patch passed
          +1 javadoc 0m 20s the patch passed with JDK v1.8.0_74
          +1 javadoc 0m 25s the patch passed with JDK v1.7.0_95
          -1 unit 70m 35s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
          -1 unit 71m 56s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          160m 31s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792021/yarn-4719-4.patch
          JIRA Issue YARN-4719
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b84b1396099e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f86850b
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10723/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10723/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 7m 26s trunk passed +1 compile 0m 31s trunk passed with JDK v1.8.0_74 +1 compile 0m 31s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 36s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 9s trunk passed +1 javadoc 0m 23s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 28s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 31s the patch passed +1 compile 0m 26s the patch passed with JDK v1.8.0_74 +1 javac 0m 26s the patch passed +1 compile 0m 28s the patch passed with JDK v1.7.0_95 -1 javac 3m 39s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2) +1 javac 0m 28s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 9 new + 304 unchanged - 8 fixed = 313 total (was 312) +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 19s the patch passed +1 javadoc 0m 20s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 25s the patch passed with JDK v1.7.0_95 -1 unit 70m 35s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. -1 unit 71m 56s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 160m 31s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792021/yarn-4719-4.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b84b1396099e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f86850b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-YARN-Build/10723/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10723/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10723/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 4s YARN-4719 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792018/yarn-4719-4.patch
          JIRA Issue YARN-4719
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10722/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 4s YARN-4719 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792018/yarn-4719-4.patch JIRA Issue YARN-4719 Console output https://builds.apache.org/job/PreCommit-YARN-Build/10722/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 7s YARN-4719 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792015/yarn-4719-4.patch
          JIRA Issue YARN-4719
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10721/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 7s YARN-4719 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792015/yarn-4719-4.patch JIRA Issue YARN-4719 Console output https://builds.apache.org/job/PreCommit-YARN-Build/10721/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          Patch that uses NodeFilter to keep non-node-tracking logic out of ClusterNodeTracker. Wangda Tan - what do you think of this approach?

          Show
          kasha Karthik Kambatla added a comment - Patch that uses NodeFilter to keep non-node-tracking logic out of ClusterNodeTracker. Wangda Tan - what do you think of this approach?
          Hide
          kasha Karthik Kambatla added a comment -

          Oh, and when the NodeFilter is null or an "IdentityFilter" (that always returns true), we could choose to skip iteration and just populate the list with the keySet(). That way, we don't need a separate method to return all nodes.

          Show
          kasha Karthik Kambatla added a comment - Oh, and when the NodeFilter is null or an "IdentityFilter" (that always returns true), we could choose to skip iteration and just populate the list with the keySet(). That way, we don't need a separate method to return all nodes.
          Hide
          kasha Karthik Kambatla added a comment -

          I may not understand about this, could you elaborate?

          When we add or remove a node, a few things are updated:

          1. The map/set holding these nodes
          2. Total cluster capacity
          3. Total inflated cluster capacity (that takes allocated but not utilized resources into account)
          4. Largest container that could potentially be allocated

          As long as it doesn't interfere with performance, I would like to keep all of this information consistent. That is if one thread has added a node, another thread shouldn't see a stale value for cluster capacity. Ensuring this consistency requires us to hold write locks when adding or removing a node, and read locks when accessing any of this information. Given that the current code has been ensuring consistency through a lock on the scheduler (at least FairScheduler), I doubt adding these read/write locks would lead to performance issues. I haven't done any benchmarking though.

          Do you agree with the approach of consistency-over-performance to begin with? If yes, we need the locks. And, making the HashMap a ConcurrentHashMap wouldn't buy us much.

          Coming to iterating over the nodes, I agree with your concern that we might proliferate ClusterNodeTracker with methods like addBlacklistedNodeIdsToList. And, understand your point that using a ConcurrentHashMap would allow us to iterate freely on the snapshot data without exposing the internals. That said, given that there are likely only a handful of cases where iterating through all the nodes is necessary and one of the goals for this library to help with preemption, node-labeling, and over-subscription, what do you think of the following construct:

            public List<NodeId> filter(NodeFilter nodeFilter) {
              List<NodeId> nodeIds = new ArrayList<>();
              readLock.lock();
              try {
                for (N node : nodes.values()) {
                  if (nodeFilter.accept(node)) {
                    nodeIds.add(node.getNodeID());
                  }
                }
              } finally {
                readLock.unlock();
              }
              return nodeIds;
            }
          

          addBlackListedNodeIdsToList can implement a trivial NodeFilter.accept as SchedulerAppUtils.isBlacklisted(app, node, LOG). We have used this approach to great effect with PathFilter.

          Thoughts?

          Show
          kasha Karthik Kambatla added a comment - I may not understand about this, could you elaborate? When we add or remove a node, a few things are updated: The map/set holding these nodes Total cluster capacity Total inflated cluster capacity (that takes allocated but not utilized resources into account) Largest container that could potentially be allocated As long as it doesn't interfere with performance, I would like to keep all of this information consistent. That is if one thread has added a node, another thread shouldn't see a stale value for cluster capacity. Ensuring this consistency requires us to hold write locks when adding or removing a node, and read locks when accessing any of this information. Given that the current code has been ensuring consistency through a lock on the scheduler (at least FairScheduler), I doubt adding these read/write locks would lead to performance issues. I haven't done any benchmarking though. Do you agree with the approach of consistency-over-performance to begin with? If yes, we need the locks. And, making the HashMap a ConcurrentHashMap wouldn't buy us much. Coming to iterating over the nodes, I agree with your concern that we might proliferate ClusterNodeTracker with methods like addBlacklistedNodeIdsToList. And, understand your point that using a ConcurrentHashMap would allow us to iterate freely on the snapshot data without exposing the internals. That said, given that there are likely only a handful of cases where iterating through all the nodes is necessary and one of the goals for this library to help with preemption, node-labeling, and over-subscription, what do you think of the following construct: public List<NodeId> filter(NodeFilter nodeFilter) { List<NodeId> nodeIds = new ArrayList<>(); readLock.lock(); try { for (N node : nodes.values()) { if (nodeFilter.accept(node)) { nodeIds.add(node.getNodeID()); } } } finally { readLock.unlock(); } return nodeIds; } addBlackListedNodeIdsToList can implement a trivial NodeFilter.accept as SchedulerAppUtils.isBlacklisted(app, node, LOG) . We have used this approach to great effect with PathFilter. Thoughts?
          Hide
          leftnoteasy Wangda Tan added a comment -

          Hi Karthik Kambatla,

          through addNode and removeNode so total_cluster_resources, total_inflated_cluster_resources (for YARN-1011), max_cluster_resources are not affected by other scheduler code.

          I may not understand about this, could you elaborate?

          For handle scheduler code to iterate nodes, we could either:

          1. Use concurrent map to avoid locking, and code will not break. Drawback: we need to handle stale data.
          2. Expose lock to external caller, so scheduler can get readlock of ClusterNodeTracker and do iteration. Drawback: iteration nodes and allocating containers could lock ClusterNodeTracker for long time.
          3. Assume synchronize lock of scheduler will be acquired when make changes to ClusterNodeTracker (like addNode, removeNode, etc.), and also when iterating nodes. We don't need extra lock of returned node collections. Drawback: this hides locks to external caller behaviors, and in the future scheduler could remove synchronized lock to get better performance.

          I would suggest to look at if #1 is doable (handle stale data and assumes eventually consistency). #1 should have best performance and flexible to future scheduler changes.

          Show
          leftnoteasy Wangda Tan added a comment - Hi Karthik Kambatla , through addNode and removeNode so total_cluster_resources, total_inflated_cluster_resources (for YARN-1011 ), max_cluster_resources are not affected by other scheduler code. I may not understand about this, could you elaborate? For handle scheduler code to iterate nodes, we could either: Use concurrent map to avoid locking, and code will not break. Drawback: we need to handle stale data. Expose lock to external caller, so scheduler can get readlock of ClusterNodeTracker and do iteration. Drawback: iteration nodes and allocating containers could lock ClusterNodeTracker for long time. Assume synchronize lock of scheduler will be acquired when make changes to ClusterNodeTracker (like addNode, removeNode, etc.), and also when iterating nodes. We don't need extra lock of returned node collections. Drawback: this hides locks to external caller behaviors, and in the future scheduler could remove synchronized lock to get better performance. I would suggest to look at if #1 is doable (handle stale data and assumes eventually consistency). #1 should have best performance and flexible to future scheduler changes.
          Hide
          kasha Karthik Kambatla added a comment -

          Wangda Tan - thanks for chiming in, you make some valid points.

          Since we are building a library for node tracking, I would like for us to restrict access to the map/set of nodes tracked only through addNode and removeNode so total_cluster_resources, total_inflated_cluster_resources (for YARN-1011), max_cluster_resources are not affected by other scheduler code. Do you think this is a reasonable goal? At least, as long as it doesn't hurt performance?

          If yes, we should decide on how to handle cases where the scheduler code needs to iterate through the nodes: (1) we could provide a snapshot copy of the map/set of nodes/nodeIds, or (2) provide a way to do the same with the right locks by adding additional methods or an abstraction (similar to lambdas) that applies to multiple methods.

          Thoughts?

          PS: By the way, thanks for pointing out the javadoc for values(). I will clean that up based on the discussion output here.

          Show
          kasha Karthik Kambatla added a comment - Wangda Tan - thanks for chiming in, you make some valid points. Since we are building a library for node tracking, I would like for us to restrict access to the map/set of nodes tracked only through addNode and removeNode so total_cluster_resources, total_inflated_cluster_resources (for YARN-1011 ), max_cluster_resources are not affected by other scheduler code. Do you think this is a reasonable goal? At least, as long as it doesn't hurt performance? If yes, we should decide on how to handle cases where the scheduler code needs to iterate through the nodes: (1) we could provide a snapshot copy of the map/set of nodes/nodeIds, or (2) provide a way to do the same with the right locks by adding additional methods or an abstraction (similar to lambdas) that applies to multiple methods. Thoughts? PS: By the way, thanks for pointing out the javadoc for values(). I will clean that up based on the discussion output here.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          +1 mvninstall 7m 59s trunk passed
          +1 compile 0m 36s trunk passed with JDK v1.8.0_72
          +1 compile 0m 33s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 24s trunk passed
          +1 mvnsite 0m 40s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 17s trunk passed
          +1 javadoc 0m 25s trunk passed with JDK v1.8.0_72
          +1 javadoc 0m 28s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 35s the patch passed
          +1 compile 0m 31s the patch passed with JDK v1.8.0_72
          +1 javac 0m 31s the patch passed
          +1 compile 0m 29s the patch passed with JDK v1.7.0_95
          -1 javac 3m 59s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2)
          +1 javac 0m 29s the patch passed
          -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 9 new + 278 unchanged - 8 fixed = 287 total (was 286)
          +1 mvnsite 0m 36s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 28s the patch passed
          +1 javadoc 0m 24s the patch passed with JDK v1.8.0_72
          +1 javadoc 0m 28s the patch passed with JDK v1.7.0_95
          -1 unit 68m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72.
          -1 unit 68m 47s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          156m 28s



          Reason Tests
          JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790991/yarn-4719-3.patch
          JIRA Issue YARN-4719
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux d84e64254cca 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 67880cc
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10687/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10687/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 7m 59s trunk passed +1 compile 0m 36s trunk passed with JDK v1.8.0_72 +1 compile 0m 33s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 24s trunk passed +1 mvnsite 0m 40s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 17s trunk passed +1 javadoc 0m 25s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 28s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 35s the patch passed +1 compile 0m 31s the patch passed with JDK v1.8.0_72 +1 javac 0m 31s the patch passed +1 compile 0m 29s the patch passed with JDK v1.7.0_95 -1 javac 3m 59s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2) +1 javac 0m 29s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 9 new + 278 unchanged - 8 fixed = 287 total (was 286) +1 mvnsite 0m 36s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 28s the patch passed +1 javadoc 0m 24s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 28s the patch passed with JDK v1.7.0_95 -1 unit 68m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. -1 unit 68m 47s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 156m 28s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790991/yarn-4719-3.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d84e64254cca 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 67880cc Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10687/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10687/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Karthik Kambatla,

          Not sure I understand the suggestion. Elaborate?

          In ver.2 patch, getAllNodes uses shallowCopy, what I meant is instead of copying the entire HashMap, you can use ConcurrentMap instead.
          In ver.3 patch, you removed shallowCopy and returns HashMap.values(), if node removed while someone is iterating values(), the behavior is undefined. See: javadoc

          I feel any logic that has to iterate through all nodes should go through ClusterNodeTracker - that way, we don't run into cases where we access the list of nodes without a lock.

          As I commented above, we can use ConcurrentMap instead of locking ClusterNodeTracker. Do you need strong consistency for addBlacklistedNodeIdsToList? (Because node list could be updated while we updating blacklistedNodes.

          Any particular reason you think this doesn't belong here?

          I would prefer to keep cleaner responsibility of ClusterNodeTracker, if we adds application logic here, we could add any logic related to SchedulerNode to this class as well. This refactoring patch is majorly for code clean up to me, I think it's better to keep it clean from the beginning.

          Show
          leftnoteasy Wangda Tan added a comment - Karthik Kambatla , Not sure I understand the suggestion. Elaborate? In ver.2 patch, getAllNodes uses shallowCopy, what I meant is instead of copying the entire HashMap, you can use ConcurrentMap instead. In ver.3 patch, you removed shallowCopy and returns HashMap.values(), if node removed while someone is iterating values(), the behavior is undefined. See: javadoc I feel any logic that has to iterate through all nodes should go through ClusterNodeTracker - that way, we don't run into cases where we access the list of nodes without a lock. As I commented above, we can use ConcurrentMap instead of locking ClusterNodeTracker. Do you need strong consistency for addBlacklistedNodeIdsToList? (Because node list could be updated while we updating blacklistedNodes. Any particular reason you think this doesn't belong here? I would prefer to keep cleaner responsibility of ClusterNodeTracker, if we adds application logic here, we could add any logic related to SchedulerNode to this class as well. This refactoring patch is majorly for code clean up to me, I think it's better to keep it clean from the beginning.
          Hide
          kasha Karthik Kambatla added a comment -

          Robert Kanter - this code touches the adaptive max allocation you have worked on. Mind taking a look to make sure I didn't screw up any of that.

          Show
          kasha Karthik Kambatla added a comment - Robert Kanter - this code touches the adaptive max allocation you have worked on. Mind taking a look to make sure I didn't screw up any of that.
          Hide
          kasha Karthik Kambatla added a comment -

          Updated patch should fix the test failures and findbugs warnings.

          Show
          kasha Karthik Kambatla added a comment - Updated patch should fix the test failures and findbugs warnings.
          Hide
          kasha Karthik Kambatla added a comment -

          For ClusterNodeTracker#nodes, can we use lock-free data structure to avoid copying the whole set?

          Not sure I understand the suggestion. Elaborate?

          We'd better not add addBlacklistedNodeIdsToList to ClusterNodeTracker since it calls application's logic, we should only include node related stuffs to ClusterNodeTracker.

          I feel any logic that has to iterate through all nodes should go through ClusterNodeTracker - that way, we don't run into cases where we access the list of nodes without a lock. Alternatively, we could get a list of nodeIDs from ClusterNodeTracker and then look up individual nodes. I am not particular about which approach, but also I don't quite see an issue with it being a part of ClusterNodeTracker. Any particular reason you think this doesn't belong here?

          Show
          kasha Karthik Kambatla added a comment - For ClusterNodeTracker#nodes, can we use lock-free data structure to avoid copying the whole set? Not sure I understand the suggestion. Elaborate? We'd better not add addBlacklistedNodeIdsToList to ClusterNodeTracker since it calls application's logic, we should only include node related stuffs to ClusterNodeTracker. I feel any logic that has to iterate through all nodes should go through ClusterNodeTracker - that way, we don't run into cases where we access the list of nodes without a lock. Alternatively, we could get a list of nodeIDs from ClusterNodeTracker and then look up individual nodes. I am not particular about which approach, but also I don't quite see an issue with it being a part of ClusterNodeTracker. Any particular reason you think this doesn't belong here?
          Hide
          leftnoteasy Wangda Tan added a comment -

          Hi Karthik Kambatla,

          Thanks for working on this patch, this is very useful.

          Took a very quick look at the patch, few comments:

          • For ClusterNodeTracker#nodes, can we use lock-free data structure to avoid copying the whole set?
          • We'd better not add addBlacklistedNodeIdsToList to ClusterNodeTracker since it calls application's logic, we should only include node related stuffs to ClusterNodeTracker.
          Show
          leftnoteasy Wangda Tan added a comment - Hi Karthik Kambatla , Thanks for working on this patch, this is very useful. Took a very quick look at the patch, few comments: For ClusterNodeTracker#nodes, can we use lock-free data structure to avoid copying the whole set? We'd better not add addBlacklistedNodeIdsToList to ClusterNodeTracker since it calls application's logic, we should only include node related stuffs to ClusterNodeTracker.
          Hide
          kasha Karthik Kambatla added a comment -

          May also be useful to expose functionality in the ClusterNodeTracker to give list of nodes in a rack, nodes that match a label expression etc. (This can possibly be another JIRA too)

          Absolutely. I wanted to move all existing common functionality into this class in this JIRA, so we can add other helper functionality in the future.

          I see that you are triggering the update thread on nodeRemoval too. I understand this might generally be useful (since the node removal might change the node ordering), but given this is a refactoring patch, maybe address that separately ?

          removeNode does a triggerUpdate today too. I just moved it a little.

          Will fix the import and the test failures here in the next iteration.

          Show
          kasha Karthik Kambatla added a comment - May also be useful to expose functionality in the ClusterNodeTracker to give list of nodes in a rack, nodes that match a label expression etc. (This can possibly be another JIRA too) Absolutely. I wanted to move all existing common functionality into this class in this JIRA, so we can add other helper functionality in the future. I see that you are triggering the update thread on nodeRemoval too. I understand this might generally be useful (since the node removal might change the node ordering), but given this is a refactoring patch, maybe address that separately ? removeNode does a triggerUpdate today too. I just moved it a little. Will fix the import and the test failures here in the next iteration.
          Hide
          asuresh Arun Suresh added a comment -

          Much needed patch Karthik Kambatla..

          Took a quick look, some comments

          1. You have an un-used import in the FairScheduler
          2. I see that you are triggering the update thread on nodeRemoval too. I understand this might generally be useful (since the node removal might change the node ordering), but given this is a refactoring patch, maybe address that separately ?
          3. May also be useful to expose functionality in the ClusterNodeTracker to give list of nodes in a rack, nodes that match a label expression etc. (This can possibly be another JIRA too)
          Show
          asuresh Arun Suresh added a comment - Much needed patch Karthik Kambatla .. Took a quick look, some comments You have an un-used import in the FairScheduler I see that you are triggering the update thread on nodeRemoval too. I understand this might generally be useful (since the node removal might change the node ordering), but given this is a refactoring patch, maybe address that separately ? May also be useful to expose functionality in the ClusterNodeTracker to give list of nodes in a rack, nodes that match a label expression etc. (This can possibly be another JIRA too)
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 6m 33s trunk passed
          +1 compile 0m 25s trunk passed with JDK v1.8.0_72
          +1 compile 0m 28s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 0m 34s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 4s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_72
          +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 29s the patch passed
          +1 compile 0m 23s the patch passed with JDK v1.8.0_72
          +1 javac 0m 23s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.7.0_95
          +1 javac 0m 27s the patch passed
          -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 12 new + 281 unchanged - 8 fixed = 293 total (was 289)
          +1 mvnsite 0m 32s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 findbugs 1m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0)
          +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72
          +1 javadoc 0m 23s the patch passed with JDK v1.7.0_95
          -1 unit 71m 5s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72.
          -1 unit 76m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 36s Patch does not generate ASF License warnings.
          164m 24s



          Reason Tests
          FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.addNode(SchedulerNode) invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:[line 71]
            Null pointer dereference of node in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:[line 140]
            Load of known null value in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:[line 140]
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) does not release lock on all exception paths At ClusterNodeTracker.java:on all exception paths At ClusterNodeTracker.java:[line 136]
            Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:[line 930]
          JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
            hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790396/yarn-4719-2.patch
          JIRA Issue YARN-4719
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e958a90eb8f0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e3ac231
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10660/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10660/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 33s trunk passed +1 compile 0m 25s trunk passed with JDK v1.8.0_72 +1 compile 0m 28s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 34s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 4s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 29s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_72 +1 javac 0m 23s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_95 +1 javac 0m 27s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 12 new + 281 unchanged - 8 fixed = 293 total (was 289) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 1m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 23s the patch passed with JDK v1.7.0_95 -1 unit 71m 5s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. -1 unit 76m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 36s Patch does not generate ASF License warnings. 164m 24s Reason Tests FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager   org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.addNode(SchedulerNode) invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java: [line 71]   Null pointer dereference of node in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java: [line 140]   Load of known null value in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java: [line 140]   org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) does not release lock on all exception paths At ClusterNodeTracker.java:on all exception paths At ClusterNodeTracker.java: [line 136]   Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java: [line 930] JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790396/yarn-4719-2.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e958a90eb8f0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e3ac231 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html unit https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10660/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10660/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10660/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 7m 5s trunk passed
          +1 compile 0m 28s trunk passed with JDK v1.8.0_72
          +1 compile 0m 28s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 0m 36s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 1m 5s trunk passed
          +1 javadoc 0m 22s trunk passed with JDK v1.8.0_72
          +1 javadoc 0m 27s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 30s the patch passed
          +1 compile 0m 23s the patch passed with JDK v1.8.0_72
          +1 javac 0m 23s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.7.0_95
          +1 javac 0m 27s the patch passed
          -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 11 new + 285 unchanged - 4 fixed = 296 total (was 289)
          +1 mvnsite 0m 32s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 findbugs 1m 16s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0)
          +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72
          +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95
          -1 unit 59m 18s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72.
          -1 unit 59m 30s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          135m 57s



          Reason Tests
          FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.addNode(SchedulerNode) invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:[line 70]
            Null pointer dereference of node in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:[line 139]
            Load of known null value in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:[line 139]
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) does not release lock on all exception paths At ClusterNodeTracker.java:on all exception paths At ClusterNodeTracker.java:[line 135]
            Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.removeNode(RMNode) Redundant null check at CapacityScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.removeNode(RMNode) Redundant null check at CapacityScheduler.java:[line 1486]
            Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.removeNode(RMNode) Redundant null check at FairScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.removeNode(RMNode) Redundant null check at FairScheduler.java:[line 903]
            Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:[line 930]
          JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestRMRestart
            hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.TestRMHA
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
            hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
            hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
          JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
            org.apache.hadoop.yarn.server.resourcemanager.TestRM
            org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestRMRestart
            hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
            hadoop.yarn.server.resourcemanager.TestRMHA
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
            hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
            hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
            org.apache.hadoop.yarn.server.resourcemanager.TestRM
            org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService
            org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790385/yarn-4719-1.patch
          JIRA Issue YARN-4719
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 13382c48ea06 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 321a80c
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10659/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10659/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 5s trunk passed +1 compile 0m 28s trunk passed with JDK v1.8.0_72 +1 compile 0m 28s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 36s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 5s trunk passed +1 javadoc 0m 22s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 30s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_72 +1 javac 0m 23s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_95 +1 javac 0m 27s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 11 new + 285 unchanged - 4 fixed = 296 total (was 289) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 1m 16s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95 -1 unit 59m 18s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. -1 unit 59m 30s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 135m 57s Reason Tests FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager   org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.addNode(SchedulerNode) invokes inefficient new Integer(int) constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java:constructor; use Integer.valueOf(int) instead At ClusterNodeTracker.java: [line 70]   Null pointer dereference of node in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) Dereferenced at ClusterNodeTracker.java: [line 139]   Load of known null value in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) At ClusterNodeTracker.java: [line 139]   org.apache.hadoop.yarn.server.resourcemanager.scheduler.ClusterNodeTracker.removeNode(NodeId) does not release lock on all exception paths At ClusterNodeTracker.java:on all exception paths At ClusterNodeTracker.java: [line 135]   Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.removeNode(RMNode) Redundant null check at CapacityScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.removeNode(RMNode) Redundant null check at CapacityScheduler.java: [line 1486]   Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.removeNode(RMNode) Redundant null check at FairScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.removeNode(RMNode) Redundant null check at FairScheduler.java: [line 903]   Redundant nullcheck of node, which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java:is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.removeNode(RMNode) Redundant null check at FifoScheduler.java: [line 930] JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.security.TestAMRMTokens   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestRMHA   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   org.apache.hadoop.yarn.server.resourcemanager.TestRM   org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.security.TestAMRMTokens   hadoop.yarn.server.resourcemanager.TestRMHA   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   org.apache.hadoop.yarn.server.resourcemanager.TestRM   org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService   org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790385/yarn-4719-1.patch JIRA Issue YARN-4719 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 13382c48ea06 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 321a80c Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html unit https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10659/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10659/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10659/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          Ran Scheduler tests locally. Fixed most failures. Ran into a couple of failures that seem like test issues. Let us see what Jenkins thinks of the new patch.

          Show
          kasha Karthik Kambatla added a comment - Ran Scheduler tests locally. Fixed most failures. Ran into a couple of failures that seem like test issues. Let us see what Jenkins thinks of the new patch.
          Hide
          kasha Karthik Kambatla added a comment -

          First patch. It removes redundancy across schedulers. Kicking off Jenkins to see what all breaks.

          Show
          kasha Karthik Kambatla added a comment - First patch. It removes redundancy across schedulers. Kicking off Jenkins to see what all breaks.

            People

            • Assignee:
              kasha Karthik Kambatla
              Reporter:
              kasha Karthik Kambatla
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development