Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4935

TestYarnClient#testSubmitIncorrectQueue fails with FairScheduler

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This test case introduced by YARN-3131 works well on CapacityScheduler but not on FairScheduler, since CS doesn't allow dynamically create a queue, while FS supports it. So if you give a random queue name, CS will reject it, but FS will create a new queue for it by default.
      One simple solution is to specific CS in this test case. /cc Chang Li. I was thinking about creating another test case for FS. But for the code introduced by YARN-3131, it may be not necessary.

      1. YARN-4935.001.patch
        1 kB
        Yufei Gu
      2. YARN-4935.002.patch
        1 kB
        Yufei Gu

        Activity

        Hide
        lichangleo Chang Li added a comment -

        agree that change in YARN-3131 is CS specific, and that change was originally driven by some problem encountered in Tez

        Show
        lichangleo Chang Li added a comment - agree that change in YARN-3131 is CS specific, and that change was originally driven by some problem encountered in Tez
        Hide
        yufeigu Yufei Gu added a comment -

        Chang Li, thanks for the quick reply and background of YARN-3131.

        Show
        yufeigu Yufei Gu added a comment - Chang Li , thanks for the quick reply and background of YARN-3131 .
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 32s trunk passed
        +1 compile 0m 15s trunk passed with JDK v1.8.0_77
        +1 compile 0m 19s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 13s trunk passed
        +1 mvnsite 0m 22s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 33s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_77
        +1 javadoc 0m 17s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 18s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.8.0_77
        +1 javac 0m 13s the patch passed
        +1 compile 0m 16s the patch passed with JDK v1.7.0_95
        +1 javac 0m 16s the patch passed
        +1 checkstyle 0m 11s the patch passed
        +1 mvnsite 0m 20s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 42s the patch passed
        +1 javadoc 0m 11s the patch passed with JDK v1.8.0_77
        +1 javadoc 0m 14s the patch passed with JDK v1.7.0_95
        -1 unit 66m 7s hadoop-yarn-client in the patch failed with JDK v1.8.0_77.
        -1 unit 66m 22s hadoop-yarn-client in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        145m 28s



        Reason Tests
        JDK v1.8.0_77 Failed junit tests hadoop.yarn.client.TestGetGroups
          hadoop.yarn.client.api.impl.TestAMRMProxy
        JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
          org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
          org.apache.hadoop.yarn.client.api.impl.TestYarnClient
          org.apache.hadoop.yarn.client.api.impl.TestNMClient
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.client.TestGetGroups
          hadoop.yarn.client.api.impl.TestAMRMProxy
        JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
          org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
          org.apache.hadoop.yarn.client.api.impl.TestYarnClient
          org.apache.hadoop.yarn.client.api.impl.TestNMClient



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797935/YARN-4935.001.patch
        JIRA Issue YARN-4935
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 38dc2cfa4eaf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1b78b2b
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11016/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11016/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11016/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11016/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11016/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11016/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 32s trunk passed +1 compile 0m 15s trunk passed with JDK v1.8.0_77 +1 compile 0m 19s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 33s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 17s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 18s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_77 +1 javac 0m 13s the patch passed +1 compile 0m 16s the patch passed with JDK v1.7.0_95 +1 javac 0m 16s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 42s the patch passed +1 javadoc 0m 11s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 14s the patch passed with JDK v1.7.0_95 -1 unit 66m 7s hadoop-yarn-client in the patch failed with JDK v1.8.0_77. -1 unit 66m 22s hadoop-yarn-client in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 145m 28s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.api.impl.TestAMRMProxy JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient JDK v1.7.0_95 Failed junit tests hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.api.impl.TestAMRMProxy JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797935/YARN-4935.001.patch JIRA Issue YARN-4935 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 38dc2cfa4eaf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1b78b2b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11016/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11016/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11016/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11016/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11016/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/11016/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        kasha Karthik Kambatla added a comment -

        If the test is capacity scheduler specific, should it be in the capacity scheduler tests? If the goal is to check the YarnClient behavior, may be the name of the test should reflect it is testing only capacity scheduler?

        Show
        kasha Karthik Kambatla added a comment - If the test is capacity scheduler specific, should it be in the capacity scheduler tests? If the goal is to check the YarnClient behavior, may be the name of the test should reflect it is testing only capacity scheduler?
        Hide
        yufeigu Yufei Gu added a comment -

        Thanks Karthik Kambatla for the review. Its goal is to test YarnClient behavior. I've change the name to testSubmitIncorrectQueueToCapacityScheduler.

        Show
        yufeigu Yufei Gu added a comment - Thanks Karthik Kambatla for the review. Its goal is to test YarnClient behavior. I've change the name to testSubmitIncorrectQueueToCapacityScheduler .
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 27s trunk passed
        +1 compile 0m 16s trunk passed with JDK v1.8.0_77
        +1 compile 0m 18s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 13s trunk passed
        +1 mvnsite 0m 21s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 33s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_77
        +1 javadoc 0m 17s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 18s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.8.0_77
        +1 javac 0m 13s the patch passed
        +1 compile 0m 16s the patch passed with JDK v1.7.0_95
        +1 javac 0m 16s the patch passed
        +1 checkstyle 0m 11s the patch passed
        +1 mvnsite 0m 19s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 41s the patch passed
        +1 javadoc 0m 12s the patch passed with JDK v1.8.0_77
        +1 javadoc 0m 14s the patch passed with JDK v1.7.0_95
        -1 unit 66m 4s hadoop-yarn-client in the patch failed with JDK v1.8.0_77.
        -1 unit 51m 31s hadoop-yarn-client in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        130m 20s



        Reason Tests
        JDK v1.8.0_77 Failed junit tests hadoop.yarn.client.TestGetGroups
          hadoop.yarn.client.api.impl.TestAMRMProxy
        JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
          org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
          org.apache.hadoop.yarn.client.api.impl.TestYarnClient
          org.apache.hadoop.yarn.client.api.impl.TestNMClient
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.client.api.impl.TestAMRMClient
          hadoop.yarn.client.TestGetGroups
          hadoop.yarn.client.api.impl.TestAMRMProxy
        JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
          org.apache.hadoop.yarn.client.api.impl.TestYarnClient
          org.apache.hadoop.yarn.client.api.impl.TestNMClient



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798060/YARN-4935.002.patch
        JIRA Issue YARN-4935
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c3af90fdfe00 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2a5da97
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11032/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11032/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11032/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11032/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11032/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11032/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 27s trunk passed +1 compile 0m 16s trunk passed with JDK v1.8.0_77 +1 compile 0m 18s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 33s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 17s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 18s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_77 +1 javac 0m 13s the patch passed +1 compile 0m 16s the patch passed with JDK v1.7.0_95 +1 javac 0m 16s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 41s the patch passed +1 javadoc 0m 12s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 14s the patch passed with JDK v1.7.0_95 -1 unit 66m 4s hadoop-yarn-client in the patch failed with JDK v1.8.0_77. -1 unit 51m 31s hadoop-yarn-client in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 130m 20s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.api.impl.TestAMRMProxy JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient JDK v1.7.0_95 Failed junit tests hadoop.yarn.client.api.impl.TestAMRMClient   hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.api.impl.TestAMRMProxy JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798060/YARN-4935.002.patch JIRA Issue YARN-4935 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c3af90fdfe00 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2a5da97 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11032/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11032/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11032/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11032/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11032/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/11032/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        yufeigu Yufei Gu added a comment -

        All failed test cases are unrelated to the latest patch after my investigating.

        Show
        yufeigu Yufei Gu added a comment - All failed test cases are unrelated to the latest patch after my investigating.
        Hide
        kasha Karthik Kambatla added a comment -

        +1, checking this in.

        Show
        kasha Karthik Kambatla added a comment - +1, checking this in.
        Hide
        kasha Karthik Kambatla added a comment -

        Thanks for reporting and fixing this, Yufei. Just committed this to trunk and branch-2.

        Show
        kasha Karthik Kambatla added a comment - Thanks for reporting and fixing this, Yufei. Just committed this to trunk and branch-2.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9642 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9642/)
        YARN-4935. TestYarnClient#testSubmitIncorrectQueue fails when (kasha: rev 6e297836d64498d90d0cd343789db2f2d490378a)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestYarnClient.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9642 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9642/ ) YARN-4935 . TestYarnClient#testSubmitIncorrectQueue fails when (kasha: rev 6e297836d64498d90d0cd343789db2f2d490378a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestYarnClient.java
        Hide
        yufeigu Yufei Gu added a comment -

        Thanks for the review and commit, Karthik Kambatla!

        Show
        yufeigu Yufei Gu added a comment - Thanks for the review and commit, Karthik Kambatla !

          People

          • Assignee:
            yufeigu Yufei Gu
            Reporter:
            yufeigu Yufei Gu
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development