Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5082

Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      see many logs like

      16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as container container_1463159225729_0002_01_000003 is not reservable
      16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as container container_1463159225729_0002_01_000004 is not reservable
      16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as container container_1463159225729_0002_01_000005 is not reservable
      16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as container container_1463159225729_0002_01_000006 is not reservable
      16/05/14 01:07:58 DEBUG fair.FSAppAttempt: Not creating reservation as container container_1463159225729_0002_01_000007 is not reservable

      1. YARN-5082.addendum.patch
        2 kB
        Arun Suresh
      2. YARN-5082.04.patch
        11 kB
        sandflee
      3. YARN-5082.03.patch
        12 kB
        sandflee
      4. YARN-5082.02.patch
        7 kB
        sandflee
      5. YARN-5082.01.patch
        6 kB
        sandflee

        Issue Links

          Activity

          Hide
          asuresh Arun Suresh added a comment -

          Vinod Kumar Vavilapalli,
          Even though the symptom is the same, the cause for the container ID leak in the FS and CS look different. In the FS case, the original issue (which this JIRA fixes) was that the containerId used to leak because a container that was created with the intent of reservation, ends up discarded due to a threshold check. From the description and comments on YARN-5074, the cause of leakage in the CS looks different.

          Show
          asuresh Arun Suresh added a comment - Vinod Kumar Vavilapalli , Even though the symptom is the same, the cause for the container ID leak in the FS and CS look different. In the FS case, the original issue (which this JIRA fixes) was that the containerId used to leak because a container that was created with the intent of reservation, ends up discarded due to a threshold check. From the description and comments on YARN-5074 , the cause of leakage in the CS looks different.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          CS has similar issue. (Linked)

          Pitching in late. Haven't looked at the patch / commit. Can this not be fixed in a unified way between the schedulers?

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - CS has similar issue. (Linked) Pitching in late. Haven't looked at the patch / commit. Can this not be fixed in a unified way between the schedulers?
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9981 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9981/)
          YARN-5082. Limit ContainerId increase in fair scheduler if the num of (kasha: rev fbbe0bb627c1b5f333eb5d5390e709cc43517420)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9981 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9981/ ) YARN-5082 . Limit ContainerId increase in fair scheduler if the num of (kasha: rev fbbe0bb627c1b5f333eb5d5390e709cc43517420) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          Hide
          kasha Karthik Kambatla added a comment -

          Just committed the addendum to trunk and branch-2.

          Thanks for fixing this, Arun Suresh.

          Show
          kasha Karthik Kambatla added a comment - Just committed the addendum to trunk and branch-2. Thanks for fixing this, Arun Suresh .
          Hide
          kasha Karthik Kambatla added a comment -

          +1. Checking this in.

          Show
          kasha Karthik Kambatla added a comment - +1. Checking this in.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 11s trunk passed
          +1 compile 0m 29s trunk passed
          +1 checkstyle 0m 23s trunk passed
          +1 mvnsite 0m 32s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 54s trunk passed
          +1 javadoc 0m 19s trunk passed
          +1 mvninstall 0m 27s the patch passed
          +1 compile 0m 28s the patch passed
          +1 javac 0m 28s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 0 new + 2 unchanged - 2 fixed = 2 total (was 4)
          +1 checkstyle 0m 19s the patch passed
          +1 mvnsite 0m 30s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 59s the patch passed
          +1 javadoc 0m 17s the patch passed
          +1 unit 31m 18s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          44m 39s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e2f6409
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811431/YARN-5082.addendum.patch
          JIRA Issue YARN-5082
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux a7bf73dd5998 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0761379
          Default Java 1.8.0_91
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12073/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12073/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 11s trunk passed +1 compile 0m 29s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 32s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 54s trunk passed +1 javadoc 0m 19s trunk passed +1 mvninstall 0m 27s the patch passed +1 compile 0m 28s the patch passed +1 javac 0m 28s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 0 new + 2 unchanged - 2 fixed = 2 total (was 4) +1 checkstyle 0m 19s the patch passed +1 mvnsite 0m 30s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 59s the patch passed +1 javadoc 0m 17s the patch passed +1 unit 31m 18s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 44m 39s Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811431/YARN-5082.addendum.patch JIRA Issue YARN-5082 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a7bf73dd5998 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0761379 Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12073/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12073/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          asuresh Arun Suresh added a comment -

          Uploading patch to move from deprecated Resource::getMemory() to Resource::getMemorySize()

          Karthik Kambatla, mind blessing this ?

          Show
          asuresh Arun Suresh added a comment - Uploading patch to move from deprecated Resource::getMemory() to Resource::getMemorySize() Karthik Kambatla , mind blessing this ?
          Hide
          asuresh Arun Suresh added a comment -

          Hmm... looks like between the last Jenkins run (which was +1 on javac) and the time the patch was committed, YARN-4844 was committed, which deprecated the Resource::getMemory() method.
          Re-opening this to add an addendum patch to fix this.

          Show
          asuresh Arun Suresh added a comment - Hmm... looks like between the last Jenkins run (which was +1 on javac) and the time the patch was committed, YARN-4844 was committed, which deprecated the Resource::getMemory() method. Re-opening this to add an addendum patch to fix this.
          Hide
          sandflee sandflee added a comment -

          Thanks Arun Suresh for reviewing and committing!

          Show
          sandflee sandflee added a comment - Thanks Arun Suresh for reviewing and committing!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9949 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9949/)
          YARN-5082. Limit ContainerId increase in fair scheduler if the num of (arun suresh: rev 5279af7cd4afb090da742a96b5786d9dee6224bc)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9949 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9949/ ) YARN-5082 . Limit ContainerId increase in fair scheduler if the num of (arun suresh: rev 5279af7cd4afb090da742a96b5786d9dee6224bc) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          Hide
          asuresh Arun Suresh added a comment -

          Apologize for the delay... Committed this to trunk and branch-2. Thanks sandflee !!

          Show
          asuresh Arun Suresh added a comment - Apologize for the delay... Committed this to trunk and branch-2. Thanks sandflee !!
          Hide
          asuresh Arun Suresh added a comment -

          LGTM, Will commit it tomorrow if there are no objections..

          Show
          asuresh Arun Suresh added a comment - LGTM, Will commit it tomorrow if there are no objections..
          Hide
          sandflee sandflee added a comment -

          TestZKRMStateStore could run locally, seems not related to this issue, file YARN-5157

          Show
          sandflee sandflee added a comment - TestZKRMStateStore could run locally, seems not related to this issue, file YARN-5157
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 23s trunk passed
          +1 compile 0m 39s trunk passed
          +1 checkstyle 0m 24s trunk passed
          +1 mvnsite 0m 43s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 33s trunk passed
          +1 javadoc 0m 25s trunk passed
          +1 mvninstall 0m 42s the patch passed
          +1 compile 0m 37s the patch passed
          +1 javac 0m 37s the patch passed
          +1 checkstyle 0m 22s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 220 unchanged - 12 fixed = 220 total (was 232)
          +1 mvnsite 0m 42s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 39s the patch passed
          +1 javadoc 0m 26s the patch passed
          -1 unit 31m 25s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          50m 42s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806160/YARN-5082.04.patch
          JIRA Issue YARN-5082
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7f761bcb8199 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9a31e5d
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11687/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11687/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 23s trunk passed +1 compile 0m 39s trunk passed +1 checkstyle 0m 24s trunk passed +1 mvnsite 0m 43s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 33s trunk passed +1 javadoc 0m 25s trunk passed +1 mvninstall 0m 42s the patch passed +1 compile 0m 37s the patch passed +1 javac 0m 37s the patch passed +1 checkstyle 0m 22s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 220 unchanged - 12 fixed = 220 total (was 232) +1 mvnsite 0m 42s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 39s the patch passed +1 javadoc 0m 26s the patch passed -1 unit 31m 25s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 50m 42s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806160/YARN-5082.04.patch JIRA Issue YARN-5082 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7f761bcb8199 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9a31e5d Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11687/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11687/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/11687/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          sandflee sandflee added a comment -

          Thanks Arun Suresh,
          1, priority was defined in FSAppAttempt as app priority, if using "Priority priority = request.getPriority()", it will generate check style warning like 'priority' hides a field. change the name of priority seems make it more clear, thought?
          2, yes, the container maybe not created yet and will be null.
          3, will do

          Show
          sandflee sandflee added a comment - Thanks Arun Suresh , 1, priority was defined in FSAppAttempt as app priority, if using "Priority priority = request.getPriority()", it will generate check style warning like 'priority' hides a field. change the name of priority seems make it more clear, thought? 2, yes, the container maybe not created yet and will be null. 3, will do
          Hide
          asuresh Arun Suresh added a comment -

          Nice catch sandflee, In general, i think the patch looks ok. Minor nits:

          1. Any reason you changed the name of the priority variable ?
          2. I am guessing you changed the argument of isReservable from 'container' to 'capability' since you have delayed creating the container until the reserve method.. can you confirm ?
          3. Maybe good to have a test case to verify that new containers are not generated.
          Show
          asuresh Arun Suresh added a comment - Nice catch sandflee , In general, i think the patch looks ok. Minor nits: Any reason you changed the name of the priority variable ? I am guessing you changed the argument of isReservable from 'container' to 'capability' since you have delayed creating the container until the reserve method.. can you confirm ? Maybe good to have a test case to verify that new containers are not generated.
          Hide
          sandflee sandflee added a comment -

          test failure are not related, cc Arun Suresh Karthik Kambatla could you help to view this?

          Show
          sandflee sandflee added a comment - test failure are not related, cc Arun Suresh Karthik Kambatla could you help to view this?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 9s trunk passed
          +1 compile 0m 28s trunk passed with JDK v1.8.0_91
          +1 compile 0m 32s trunk passed with JDK v1.7.0_101
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 0m 37s trunk passed
          +1 mvneclipse 0m 17s trunk passed
          +1 findbugs 1m 8s trunk passed
          +1 javadoc 0m 24s trunk passed with JDK v1.8.0_91
          +1 javadoc 0m 30s trunk passed with JDK v1.7.0_101
          +1 mvninstall 0m 32s the patch passed
          +1 compile 0m 26s the patch passed with JDK v1.8.0_91
          +1 javac 0m 26s the patch passed
          +1 compile 0m 29s the patch passed with JDK v1.7.0_101
          +1 javac 0m 29s the patch passed
          +1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 13 unchanged - 12 fixed = 13 total (was 25)
          +1 mvnsite 0m 33s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 17s the patch passed
          +1 javadoc 0m 20s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 25s the patch passed with JDK v1.7.0_101
          -1 unit 34m 10s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
          -1 unit 37m 43s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101.
          +1 asflicense 0m 28s Patch does not generate ASF License warnings.
          89m 42s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804025/YARN-5082.02.patch
          JIRA Issue YARN-5082
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 3971642210c4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3fa1380
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11467/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11467/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11467/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11467/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11467/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11467/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 9s trunk passed +1 compile 0m 28s trunk passed with JDK v1.8.0_91 +1 compile 0m 32s trunk passed with JDK v1.7.0_101 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 37s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 1m 8s trunk passed +1 javadoc 0m 24s trunk passed with JDK v1.8.0_91 +1 javadoc 0m 30s trunk passed with JDK v1.7.0_101 +1 mvninstall 0m 32s the patch passed +1 compile 0m 26s the patch passed with JDK v1.8.0_91 +1 javac 0m 26s the patch passed +1 compile 0m 29s the patch passed with JDK v1.7.0_101 +1 javac 0m 29s the patch passed +1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 13 unchanged - 12 fixed = 13 total (was 25) +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 17s the patch passed +1 javadoc 0m 20s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 25s the patch passed with JDK v1.7.0_101 -1 unit 34m 10s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. -1 unit 37m 43s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101. +1 asflicense 0m 28s Patch does not generate ASF License warnings. 89m 42s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804025/YARN-5082.02.patch JIRA Issue YARN-5082 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3971642210c4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3fa1380 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11467/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11467/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11467/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11467/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11467/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/11467/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          sandflee sandflee added a comment -

          rename FSAppAttempt.priority to FSAppAttempt.appPriority to fix checkstyle warning

          Show
          sandflee sandflee added a comment - rename FSAppAttempt.priority to FSAppAttempt.appPriority to fix checkstyle warning
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 55s trunk passed
          +1 compile 0m 38s trunk passed with JDK v1.8.0_91
          +1 compile 0m 35s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 24s trunk passed
          +1 mvnsite 0m 41s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 findbugs 1m 19s trunk passed
          +1 javadoc 0m 25s trunk passed with JDK v1.8.0_91
          +1 javadoc 0m 31s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 36s the patch passed
          +1 compile 0m 33s the patch passed with JDK v1.8.0_91
          +1 javac 0m 33s the patch passed
          +1 compile 0m 30s the patch passed with JDK v1.7.0_95
          +1 javac 0m 30s the patch passed
          -1 checkstyle 0m 22s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 5 new + 24 unchanged - 1 fixed = 29 total (was 25)
          +1 mvnsite 0m 36s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 30s the patch passed
          +1 javadoc 0m 24s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 27s the patch passed with JDK v1.7.0_95
          -1 unit 31m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
          -1 unit 30m 36s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 18s Patch does not generate ASF License warnings.
          81m 41s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803989/YARN-5082.01.patch
          JIRA Issue YARN-5082
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 34928fc6dfba 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3fa1380
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11460/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11460/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11460/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11460/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11460/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11460/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11460/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 55s trunk passed +1 compile 0m 38s trunk passed with JDK v1.8.0_91 +1 compile 0m 35s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 24s trunk passed +1 mvnsite 0m 41s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 1m 19s trunk passed +1 javadoc 0m 25s trunk passed with JDK v1.8.0_91 +1 javadoc 0m 31s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 36s the patch passed +1 compile 0m 33s the patch passed with JDK v1.8.0_91 +1 javac 0m 33s the patch passed +1 compile 0m 30s the patch passed with JDK v1.7.0_95 +1 javac 0m 30s the patch passed -1 checkstyle 0m 22s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 5 new + 24 unchanged - 1 fixed = 29 total (was 25) +1 mvnsite 0m 36s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 30s the patch passed +1 javadoc 0m 24s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 27s the patch passed with JDK v1.7.0_95 -1 unit 31m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. -1 unit 30m 36s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 81m 41s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803989/YARN-5082.01.patch JIRA Issue YARN-5082 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 34928fc6dfba 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3fa1380 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11460/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11460/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11460/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11460/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11460/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11460/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/11460/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          CS has similar issue. (Linked)

          Show
          leftnoteasy Wangda Tan added a comment - CS has similar issue. (Linked)
          Hide
          sandflee sandflee added a comment -

          we enable continuous scheduling, so containerID increased (corresponding to a container creation) at every schedule period. create a container only when container could be allocated or container reserved at first time seems more reasonable.

          Show
          sandflee sandflee added a comment - we enable continuous scheduling, so containerID increased (corresponding to a container creation) at every schedule period. create a container only when container could be allocated or container reserved at first time seems more reasonable.

            People

            • Assignee:
              sandflee sandflee
              Reporter:
              sandflee sandflee
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development