Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6165

Intra-queue preemption occurs even when preemption is turned off for a specific queue.

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Intra-queue preemption occurs even when preemption is turned on for the whole cluster (yarn.resourcemanager.scheduler.monitor.enable == true) but turned off for a specific queue (yarn.scheduler.capacity.root.queue1.disable_preemption == true).

        Activity

        Hide
        eepayne Eric Payne added a comment -

        Use Case:

        • Configure queues with cluster-wide preemption on, but a specific queue's preemption off (see above).
        • Submit a job at priority 1 that fills the entire queue
        • Submit a job as the same user at priority 2

        Containers from the first job will be preempted when they shouldn't be.

        Show
        eepayne Eric Payne added a comment - Use Case: Configure queues with cluster-wide preemption on, but a specific queue's preemption off (see above). Submit a job at priority 1 that fills the entire queue Submit a job as the same user at priority 2 Containers from the first job will be preempted when they shouldn't be.
        Hide
        eepayne Eric Payne added a comment -

        IntraQueueCandidatesSelector#selectCandidates was not checking the enable/disable status of each queue prior to calculating the resToObtainByPartition.

        To test this, I manually modified each of the tests in TestProportionalCapacityPreemptionPolicyIntraQueue to disable preemption on all of the test queues. Without this fix, the tests passed when they should have failed. Meaning, with the preemption disabled on all of the test queues, the tests continued to select containers for preemption. However, when I added this fix, those modified tests started to fail.

        Sunil G, Wangda Tan, and Jason Lowe, any comments would be greatly appreciated.

        Show
        eepayne Eric Payne added a comment - IntraQueueCandidatesSelector#selectCandidates was not checking the enable/disable status of each queue prior to calculating the resToObtainByPartition . To test this, I manually modified each of the tests in TestProportionalCapacityPreemptionPolicyIntraQueue to disable preemption on all of the test queues. Without this fix, the tests passed when they should have failed. Meaning, with the preemption disabled on all of the test queues, the tests continued to select containers for preemption. However, when I added this fix, those modified tests started to fail. Sunil G , Wangda Tan , and Jason Lowe , any comments would be greatly appreciated.
        Hide
        sunilg Sunil G added a comment -

        Thanks Eric Payne. I think patch looks fine for me.

        Show
        sunilg Sunil G added a comment - Thanks Eric Payne . I think patch looks fine for me.
        Hide
        eepayne Eric Payne added a comment -

        I think patch looks fine for me.

        Thanks Sunil G. Will you be committing this?

        Show
        eepayne Eric Payne added a comment - I think patch looks fine for me. Thanks Sunil G . Will you be committing this?
        Hide
        jlowe Jason Lowe added a comment -

        I'd like to take a quick look, and this needs a Jenkins run anyway.

        Show
        jlowe Jason Lowe added a comment - I'd like to take a quick look, and this needs a Jenkins run anyway.
        Hide
        eepayne Eric Payne added a comment -

        this needs a Jenkins run anyway

        Thanks Jason Lowe. I kicked the build.

        Show
        eepayne Eric Payne added a comment - this needs a Jenkins run anyway Thanks Jason Lowe . I kicked the build.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        -1 mvninstall 4m 30s root in trunk failed.
        +1 compile 0m 31s trunk passed
        +1 checkstyle 0m 24s trunk passed
        +1 mvnsite 0m 32s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 56s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 35s the patch passed
        +1 compile 0m 34s the patch passed
        +1 javac 0m 34s the patch passed
        +1 checkstyle 0m 28s the patch passed
        +1 mvnsite 0m 35s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        +1 findbugs 1m 13s the patch passed
        +1 javadoc 0m 19s the patch passed
        -1 unit 40m 13s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        53m 27s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6165
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855987/YARN-6165.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f9fdf9ee8081 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 241c1cc
        Default Java 1.8.0_121
        mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/15209/artifact/patchprocess/branch-mvninstall-root.txt
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15209/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/15209/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15209/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/15209/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. -1 mvninstall 4m 30s root in trunk failed. +1 compile 0m 31s trunk passed +1 checkstyle 0m 24s trunk passed +1 mvnsite 0m 32s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 56s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 35s the patch passed +1 compile 0m 34s the patch passed +1 javac 0m 34s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 0m 35s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 1m 13s the patch passed +1 javadoc 0m 19s the patch passed -1 unit 40m 13s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 53m 27s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6165 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855987/YARN-6165.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f9fdf9ee8081 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 241c1cc Default Java 1.8.0_121 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/15209/artifact/patchprocess/branch-mvninstall-root.txt findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15209/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15209/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15209/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/15209/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        -1 mvninstall 4m 45s root in trunk failed.
        +1 compile 0m 40s trunk passed
        +1 checkstyle 0m 30s trunk passed
        +1 mvnsite 0m 40s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 1m 14s trunk passed
        +1 javadoc 0m 23s trunk passed
        +1 mvninstall 0m 42s the patch passed
        +1 compile 0m 36s the patch passed
        +1 javac 0m 36s the patch passed
        +1 checkstyle 0m 25s the patch passed
        +1 mvnsite 0m 40s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        +1 findbugs 1m 18s the patch passed
        +1 javadoc 0m 27s the patch passed
        -1 unit 43m 9s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        58m 2s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6165
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855987/YARN-6165.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 2d24ec6a2e2a 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 241c1cc
        Default Java 1.8.0_121
        mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/15210/artifact/patchprocess/branch-mvninstall-root.txt
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15210/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/15210/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15210/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/15210/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. -1 mvninstall 4m 45s root in trunk failed. +1 compile 0m 40s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 0m 40s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 14s trunk passed +1 javadoc 0m 23s trunk passed +1 mvninstall 0m 42s the patch passed +1 compile 0m 36s the patch passed +1 javac 0m 36s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 0m 40s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 1m 18s the patch passed +1 javadoc 0m 27s the patch passed -1 unit 43m 9s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 58m 2s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6165 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855987/YARN-6165.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2d24ec6a2e2a 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 241c1cc Default Java 1.8.0_121 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/15210/artifact/patchprocess/branch-mvninstall-root.txt findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15210/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15210/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15210/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/15210/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jlowe Jason Lowe added a comment -

        +1 lgtm. The TestRMRestart failure is unrelated and will be fixed by YARN-5548. I'll fixup the whitespace nit during the commit.

        Show
        jlowe Jason Lowe added a comment - +1 lgtm. The TestRMRestart failure is unrelated and will be fixed by YARN-5548 . I'll fixup the whitespace nit during the commit.
        Hide
        jlowe Jason Lowe added a comment -

        Thanks to Eric for the contribution and to Sunil for additional review! I committed this to trunk, branch-2, and branch-2.8.

        Show
        jlowe Jason Lowe added a comment - Thanks to Eric for the contribution and to Sunil for additional review! I committed this to trunk, branch-2, and branch-2.8.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11375 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11375/)
        YARN-6165. Intra-queue preemption occurs even when preemption is turned (jlowe: rev d7762a55113a529abd6f4ecb8e6d9b0a84b56e08)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicyIntraQueue.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/IntraQueueCandidatesSelector.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11375 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11375/ ) YARN-6165 . Intra-queue preemption occurs even when preemption is turned (jlowe: rev d7762a55113a529abd6f4ecb8e6d9b0a84b56e08) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicyIntraQueue.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/IntraQueueCandidatesSelector.java
        Hide
        eepayne Eric Payne added a comment -

        Thanks Jason and Sunil.

        Show
        eepayne Eric Payne added a comment - Thanks Jason and Sunil.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

          People

          • Assignee:
            eepayne Eric Payne
            Reporter:
            eepayne Eric Payne
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development