Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4465

SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Disable label from rm side yarn.nodelabel.enable=false
      Capacity scheduler label configuration for queue is available as below
      default label for queue = b1 as 3 and accessible labels as 1,3
      Submit application to queue A .

      Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException): Invalid resource request, queue=b1 doesn't have permission to access all labels in resource request. labelExpression of resource request=3. Queue labels=1,3
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:304)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:234)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:216)
              at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.validateAndCreateResourceRequest(RMAppManager.java:401)
              at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:340)
              at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:283)
              at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:602)
              at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:247)
      
      
      1. Ignore default label expression when label is disabled or
      2. NormalizeResourceRequest we can set label expression to <DEFAULT PARTITION> when node label is not enabled or
      3. Improve message
      1. 0001-YARN-4465.patch
        14 kB
        Bibin A Chundatt
      2. 0002-YARN-4465.patch
        16 kB
        Bibin A Chundatt
      3. 0003-YARN-4465.patch
        16 kB
        Bibin A Chundatt
      4. 0004-YARN-4465.patch
        9 kB
        Bibin A Chundatt
      5. 0006-YARN-4465.patch
        10 kB
        Bibin A Chundatt
      6. 0007-YARN-4465.patch
        9 kB
        Bibin A Chundatt

        Issue Links

          Activity

          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Hi Wangda Tan/Sunil G any thoughts ?

          The message is confusing resource request=3. Queue labels=1,3 request was for label 3 and labels available on queue are 1,3.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Hi Wangda Tan / Sunil G any thoughts ? The message is confusing resource request=3. Queue labels=1,3 request was for label 3 and labels available on queue are 1,3.
          Hide
          sunilg Sunil G added a comment -

          Bibin A Chundatt Agree that message is not clear here. However this is a negative scenario where feature is disabled from top level, and few configuration is made. I feel we could reset such expression in resource request(such as explicit label from request OR default expression in queue) to a common agreed default label. This is harmless, however we are changing some default behavior hence proper logging is needed to inform user/admin that a default behavior is took place. Wangda Tan thoughts?

          Show
          sunilg Sunil G added a comment - Bibin A Chundatt Agree that message is not clear here. However this is a negative scenario where feature is disabled from top level, and few configuration is made. I feel we could reset such expression in resource request(such as explicit label from request OR default expression in queue) to a common agreed default label. This is harmless, however we are changing some default behavior hence proper logging is needed to inform user/admin that a default behavior is took place. Wangda Tan thoughts?
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks Bibin A Chundatt/Sunil G for looking at this issue.

          I agree most of what Sunil G said, this is a mis configuration, it's fine to report error, but we need to improve message. And change default behavior (like ignore default node label expression) could hide mis configurations which should be fixed.

          How about throw exception with different message when labels are not found in the cluster?

                  if (null != rmContext) {
                    RMNodeLabelsManager nlm = rmContext.getNodeLabelManager();
                    if (nlm != null && !nlm.containsNodeLabel(str)) {
                      return false;
                    }
                  }
          
          Show
          leftnoteasy Wangda Tan added a comment - Thanks Bibin A Chundatt / Sunil G for looking at this issue. I agree most of what Sunil G said, this is a mis configuration, it's fine to report error, but we need to improve message. And change default behavior (like ignore default node label expression) could hide mis configurations which should be fixed. How about throw exception with different message when labels are not found in the cluster? if ( null != rmContext) { RMNodeLabelsManager nlm = rmContext.getNodeLabelManager(); if (nlm != null && !nlm.containsNodeLabel(str)) { return false ; } }
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Hi Wangda Tan/Sunil G
          Thank you for looking into the issue

          The below 2 points will fix as part of this jira

          1. labels are not found in the cluster thrown exception with message
          2. SchedulerUtils.normalizeAndValidateRequest when node label is not enabled will reset the labels to default with proper logging.

          Thoughts??

          Show
          bibinchundatt Bibin A Chundatt added a comment - Hi Wangda Tan / Sunil G Thank you for looking into the issue The below 2 points will fix as part of this jira labels are not found in the cluster thrown exception with message SchedulerUtils.normalizeAndValidateRequest when node label is not enabled will reset the labels to default with proper logging. Thoughts??
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Wangda Tan/Sunil G

          Hope only the above 2 changes are required. Will update the patch soon.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Wangda Tan / Sunil G Hope only the above 2 changes are required. Will update the patch soon.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Hi Sunil G/Wangda Tan
          Attaching patch for review. Patch updates the below

          1. Typo correction for message
          2. If cluster node labels are not available throw InvalidLabelresourceexception
          3. Reset request label if Node Labels are not enabled
          Show
          bibinchundatt Bibin A Chundatt added a comment - Hi Sunil G / Wangda Tan Attaching patch for review. Patch updates the below Typo correction for message If cluster node labels are not available throw InvalidLabelresourceexception Reset request label if Node Labels are not enabled
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 51s trunk passed
          +1 compile 0m 29s trunk passed with JDK v1.8.0_66
          +1 compile 0m 32s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 14s trunk passed
          +1 mvnsite 0m 38s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 1m 13s trunk passed
          +1 javadoc 0m 23s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 30s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 33s the patch passed
          +1 compile 0m 26s the patch passed with JDK v1.8.0_66
          +1 javac 0m 26s the patch passed
          +1 compile 0m 30s the patch passed with JDK v1.7.0_91
          +1 javac 0m 30s the patch passed
          +1 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 75 unchanged - 2 fixed = 75 total (was 77)
          +1 mvnsite 0m 36s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 23s the patch passed
          +1 javadoc 0m 19s the patch passed with JDK v1.8.0_66
          -1 javadoc 2m 40s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
          +1 javadoc 0m 26s the patch passed with JDK v1.7.0_91
          -1 unit 65m 4s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 65m 37s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          148m 53s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782900/0001-YARN-4465.patch
          JIRA Issue YARN-4465
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 9e6cdbb207fb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d40859f
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-YARN-Build/10320/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10320/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10320/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10320/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10320/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10320/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Max memory used 77MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10320/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 51s trunk passed +1 compile 0m 29s trunk passed with JDK v1.8.0_66 +1 compile 0m 32s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 38s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 13s trunk passed +1 javadoc 0m 23s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 30s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 33s the patch passed +1 compile 0m 26s the patch passed with JDK v1.8.0_66 +1 javac 0m 26s the patch passed +1 compile 0m 30s the patch passed with JDK v1.7.0_91 +1 javac 0m 30s the patch passed +1 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 75 unchanged - 2 fixed = 75 total (was 77) +1 mvnsite 0m 36s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 23s the patch passed +1 javadoc 0m 19s the patch passed with JDK v1.8.0_66 -1 javadoc 2m 40s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) +1 javadoc 0m 26s the patch passed with JDK v1.7.0_91 -1 unit 65m 4s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 65m 37s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 148m 53s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782900/0001-YARN-4465.patch JIRA Issue YARN-4465 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9e6cdbb207fb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d40859f Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-YARN-Build/10320/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10320/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10320/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10320/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10320/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10320/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10320/console This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Hi Bibin A Chundatt
          Thanks for working on this, some comments:
          1)

            public static void normalizeAndValidateRequests(List<ResourceRequest> ask,
                Resource maximumResource, String queueName, YarnScheduler scheduler,
                RMContext rmContext)
          

          You can avoid get queue info if node label is disabled. Same to

            public static void normalizeAndValidateRequest(ResourceRequest resReq,
                Resource maximumResource, String queueName, YarnScheduler scheduler,
                boolean isRecovery, RMContext rmContext, QueueInfo queueInfo)
          

          2) In

            private static void normalizeNodeLabelExpressionInRequest(
                ResourceRequest resReq, QueueInfo queueInfo, RMContext rmContext) {
          

          Instead of passing rmContext as parameter, you can pass a boolean to indicate if node label is enabled.

          3)
          LOG.debug("Resetti should be wrapped by "isDebugEnabled"

          4) In

            private static void validateResourceRequest(ResourceRequest resReq,
                Resource maximumResource, QueueInfo queueInfo, RMContext rmContext)
          

          You can use a "if (nodeLabelEnabled) {...}" to wrap code after

          String labelExp = resReq.getNodeLabelExpression();
          ...
          
          Show
          leftnoteasy Wangda Tan added a comment - Hi Bibin A Chundatt Thanks for working on this, some comments: 1) public static void normalizeAndValidateRequests(List<ResourceRequest> ask, Resource maximumResource, String queueName, YarnScheduler scheduler, RMContext rmContext) You can avoid get queue info if node label is disabled. Same to public static void normalizeAndValidateRequest(ResourceRequest resReq, Resource maximumResource, String queueName, YarnScheduler scheduler, boolean isRecovery, RMContext rmContext, QueueInfo queueInfo) 2) In private static void normalizeNodeLabelExpressionInRequest( ResourceRequest resReq, QueueInfo queueInfo, RMContext rmContext) { Instead of passing rmContext as parameter, you can pass a boolean to indicate if node label is enabled. 3) LOG.debug("Resetti should be wrapped by "isDebugEnabled" 4) In private static void validateResourceRequest(ResourceRequest resReq, Resource maximumResource, QueueInfo queueInfo, RMContext rmContext) You can use a "if (nodeLabelEnabled) {...}" to wrap code after String labelExp = resReq.getNodeLabelExpression(); ...
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Wangda Tan
          Thank you for reviewing patch. Uploaded latest patch after correcttion

          Show
          bibinchundatt Bibin A Chundatt added a comment - Wangda Tan Thank you for reviewing patch. Uploaded latest patch after correcttion
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Attached patch after fixing findbug issue

          Show
          bibinchundatt Bibin A Chundatt added a comment - Attached patch after fixing findbug issue
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 41s trunk passed
          +1 compile 0m 27s trunk passed with JDK v1.8.0_66
          +1 compile 0m 31s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 14s trunk passed
          +1 mvnsite 0m 36s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 13s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 27s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 30s the patch passed
          +1 compile 0m 24s the patch passed with JDK v1.8.0_66
          +1 javac 0m 24s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.7.0_91
          +1 javac 0m 27s the patch passed
          -1 checkstyle 0m 14s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 74 unchanged - 3 fixed = 76 total (was 77)
          +1 mvnsite 0m 33s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 21s the patch passed
          +1 javadoc 0m 18s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 23s the patch passed with JDK v1.7.0_91
          -1 unit 60m 28s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 61m 17s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 18s Patch does not generate ASF License warnings.
          150m 22s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783264/0003-YARN-4465.patch
          JIRA Issue YARN-4465
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux cb9e173d09d1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1acc509
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10337/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10337/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10337/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10337/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10337/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10337/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Max memory used 77MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10337/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 41s trunk passed +1 compile 0m 27s trunk passed with JDK v1.8.0_66 +1 compile 0m 31s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 36s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 13s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 30s the patch passed +1 compile 0m 24s the patch passed with JDK v1.8.0_66 +1 javac 0m 24s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_91 +1 javac 0m 27s the patch passed -1 checkstyle 0m 14s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 2 new + 74 unchanged - 3 fixed = 76 total (was 77) +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 21s the patch passed +1 javadoc 0m 18s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 23s the patch passed with JDK v1.7.0_91 -1 unit 60m 28s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 61m 17s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 150m 22s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783264/0003-YARN-4465.patch JIRA Issue YARN-4465 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux cb9e173d09d1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1acc509 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10337/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10337/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10337/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10337/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10337/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10337/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10337/console This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Hi Bibin A Chundatt,

          I felt I made a mistake for previous comment, when the label is not enabled, we shouldn't simply ignore checking labels in request. Request with label will be starving when labels are disabled. Reset such labels to empty is also problematic, we should let application knows about the failure.

          Here is my proposal, when label is disabled:

          • Any request with label will be rejected (throws invalid label exception)
          • Queues with label configured will be rejected, including default-node-label-expression, accessible-node-labels (accessible-node-labels is * by default, but we shouldn't allow explicitly set accessible-node-labels to non-empty and non *), and label-related capacities (check org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueUtils#loadCapacitiesByLabelsFromConf)

          Queue related changes could be done in a separated JIRA.

          Thoughts?

          Show
          leftnoteasy Wangda Tan added a comment - Hi Bibin A Chundatt , I felt I made a mistake for previous comment, when the label is not enabled, we shouldn't simply ignore checking labels in request. Request with label will be starving when labels are disabled. Reset such labels to empty is also problematic, we should let application knows about the failure. Here is my proposal, when label is disabled: Any request with label will be rejected (throws invalid label exception) Queues with label configured will be rejected, including default-node-label-expression, accessible-node-labels (accessible-node-labels is * by default, but we shouldn't allow explicitly set accessible-node-labels to non-empty and non *), and label-related capacities (check org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueUtils#loadCapacitiesByLabelsFromConf ) Queue related changes could be done in a separated JIRA. Thoughts?
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Hi Wangda Tan
          Thank you for review.

          In this jira the below will handle

          1. SchedulerUtils#normalizeAndValidateRequest if label not enabled and request with label will throw InvalidLabelResourceRequestException
          2. In RMlabelmanager if label not available then throw InvalidLabelResourceRequestException with proper exception
            Will update the above and upload patch soon.
          Show
          bibinchundatt Bibin A Chundatt added a comment - Hi Wangda Tan Thank you for review. In this jira the below will handle SchedulerUtils#normalizeAndValidateRequest if label not enabled and request with label will throw InvalidLabelResourceRequestException In RMlabelmanager if label not available then throw InvalidLabelResourceRequestException with proper exception Will update the above and upload patch soon.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Attaching patch for review.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Attaching patch for review.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 50s trunk passed
          +1 compile 0m 29s trunk passed with JDK v1.8.0_66
          +1 compile 0m 31s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 17s trunk passed
          +1 mvnsite 0m 36s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 13s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 27s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 33s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.8.0_66
          +1 javac 0m 27s the patch passed
          +1 compile 0m 29s the patch passed with JDK v1.7.0_91
          +1 javac 0m 29s the patch passed
          +1 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 20 unchanged - 1 fixed = 20 total (was 21)
          +1 mvnsite 0m 34s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 23s the patch passed
          +1 javadoc 0m 20s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 25s the patch passed with JDK v1.7.0_91
          -1 unit 65m 17s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 65m 48s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          148m 9s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.TestResourceManager
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
            hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestClientRMService
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.TestResourceManager
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
            hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestClientRMService
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785168/0004-YARN-4465.patch
          JIRA Issue YARN-4465
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7df66a5aadf0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a277bdc
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10435/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10435/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10435/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10435/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10435/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10435/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 50s trunk passed +1 compile 0m 29s trunk passed with JDK v1.8.0_66 +1 compile 0m 31s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 36s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 13s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 33s the patch passed +1 compile 0m 27s the patch passed with JDK v1.8.0_66 +1 javac 0m 27s the patch passed +1 compile 0m 29s the patch passed with JDK v1.7.0_91 +1 javac 0m 29s the patch passed +1 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 20 unchanged - 1 fixed = 20 total (was 21) +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 23s the patch passed +1 javadoc 0m 20s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 25s the patch passed with JDK v1.7.0_91 -1 unit 65m 17s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 65m 48s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 148m 9s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   hadoop.yarn.server.resourcemanager.TestResourceManager   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestClientRMService   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   hadoop.yarn.server.resourcemanager.TestResourceManager   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestClientRMService   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785168/0004-YARN-4465.patch JIRA Issue YARN-4465 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7df66a5aadf0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a277bdc Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/10435/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10435/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10435/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10435/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10435/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10435/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Hi Bibin A Chundatt

          Thanks for updating patch. Few comments.:

          1. Comment mentioned in normalizeAndValidateRequest is not correct. Pls change.

          +    // If Node label is not enabled reset request to no label
          +    if (null != conf && !YarnConfiguration.areNodeLabelsEnabled(conf)) {
          +

          2. I have a doubt

              // we don't allow specify label expression with more than one node labels now
              if (labelExp != null && labelExp.contains("&&")) {
          

          In validateResourceRequest, if labelExp contains "&&", exception is thrown.
          So do we need to handle special character in checkQueueLabelInLabelManager ?

          3. In checkQueueLabelInLabelManager, we can remove the statement called str = str.trim();
          4. test cases are asserting with InvalidResourceRequestException, but this patch is now modified to throw InvalidLabelResourceRequestException. Tests will pass, but I think we can check this correct exception itself, OR am i missing something. pls help to correct me.

          Show
          sunilg Sunil G added a comment - Hi Bibin A Chundatt Thanks for updating patch. Few comments.: 1. Comment mentioned in normalizeAndValidateRequest is not correct. Pls change. + // If Node label is not enabled reset request to no label + if (null != conf && !YarnConfiguration.areNodeLabelsEnabled(conf)) { + 2. I have a doubt // we don't allow specify label expression with more than one node labels now if (labelExp != null && labelExp.contains( "&&" )) { In validateResourceRequest, if labelExp contains "&&", exception is thrown. So do we need to handle special character in checkQueueLabelInLabelManager ? 3. In checkQueueLabelInLabelManager , we can remove the statement called str = str.trim(); 4. test cases are asserting with InvalidResourceRequestException , but this patch is now modified to throw InvalidLabelResourceRequestException . Tests will pass, but I think we can check this correct exception itself, OR am i missing something. pls help to correct me.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Sunil G
          Thank you for reviewing patch.
          Point 4: Havnt handled since the exception thrown from normlize is ResourceException.
          Attaching latest patch

          Show
          bibinchundatt Bibin A Chundatt added a comment - Sunil G Thank you for reviewing patch. Point 4: Havnt handled since the exception thrown from normlize is ResourceException. Attaching latest patch
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Uploading patch to trigger Jenkins again

          Show
          bibinchundatt Bibin A Chundatt added a comment - Uploading patch to trigger Jenkins again
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 6m 38s trunk passed
          +1 compile 0m 26s trunk passed with JDK v1.8.0_72
          +1 compile 0m 29s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 18s trunk passed
          +1 mvnsite 0m 37s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 7s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_72
          +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 32s the patch passed
          +1 compile 0m 23s the patch passed with JDK v1.8.0_72
          +1 javac 0m 23s the patch passed
          +1 compile 0m 26s the patch passed with JDK v1.7.0_95
          +1 javac 0m 26s the patch passed
          +1 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 20 unchanged - 1 fixed = 20 total (was 21)
          +1 mvnsite 0m 33s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 17s the patch passed
          +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72
          +1 javadoc 0m 25s the patch passed with JDK v1.7.0_95
          -1 unit 71m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72.
          -1 unit 71m 59s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 18s Patch does not generate ASF License warnings.
          159m 55s



          Reason Tests
          JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788728/0006-YARN-4465.patch
          JIRA Issue YARN-4465
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c924a0549b58 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b68901d
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10597/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10597/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10597/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10597/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10597/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10597/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 38s trunk passed +1 compile 0m 26s trunk passed with JDK v1.8.0_72 +1 compile 0m 29s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 37s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 7s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 32s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_72 +1 javac 0m 23s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_95 +1 javac 0m 26s the patch passed +1 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 20 unchanged - 1 fixed = 20 total (was 21) +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 17s the patch passed +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 25s the patch passed with JDK v1.7.0_95 -1 unit 71m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. -1 unit 71m 59s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 159m 55s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788728/0006-YARN-4465.patch JIRA Issue YARN-4465 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c924a0549b58 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b68901d Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/10597/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10597/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10597/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10597/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10597/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10597/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks Bibin A Chundatt,

          Generally looks good, one minor comment:
          Could you avoid show all cluster labels in exception when label doesn't exist in cluster?

          337	      if (nlm != null && !nlm.containsNodeLabel(labelExpression)) {
          338	        throw new InvalidLabelResourceRequestException(
          339	            "Invalid label resource request, cluster do not contain "
          340	                + ", label= " + labelExpression + " Cluster labels= "
          341	                + StringUtils.join(nlm.getClusterNodeLabels(), ','));
          315	      }
          342	      }
          

          A cluster could have many labels.

          And could you explain why change of TestKillApplicationWithRMHA is needed?

          Show
          leftnoteasy Wangda Tan added a comment - Thanks Bibin A Chundatt , Generally looks good, one minor comment: Could you avoid show all cluster labels in exception when label doesn't exist in cluster? 337 if (nlm != null && !nlm.containsNodeLabel(labelExpression)) { 338 throw new InvalidLabelResourceRequestException( 339 "Invalid label resource request, cluster do not contain " 340 + ", label= " + labelExpression + " Cluster labels= " 341 + StringUtils.join(nlm.getClusterNodeLabels(), ',')); 315 } 342 } A cluster could have many labels. And could you explain why change of TestKillApplicationWithRMHA is needed?
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Wangda Tan
          Thank you for review comments

          Could you avoid show all cluster labels in exception when label doesn't exist in cluster?

          Done

          And could you explain why change of TestKillApplicationWithRMHA is needed?

          Sorry had done the update for debugging testcase, excluded the updation from latest patch .

          Show
          bibinchundatt Bibin A Chundatt added a comment - Wangda Tan Thank you for review comments Could you avoid show all cluster labels in exception when label doesn't exist in cluster? Done And could you explain why change of TestKillApplicationWithRMHA is needed? Sorry had done the update for debugging testcase, excluded the updation from latest patch .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 43s trunk passed
          +1 compile 0m 27s trunk passed with JDK v1.8.0_72
          +1 compile 0m 29s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 17s trunk passed
          +1 mvnsite 0m 35s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 4s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_72
          +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 29s the patch passed
          +1 compile 0m 24s the patch passed with JDK v1.8.0_72
          +1 javac 0m 24s the patch passed
          +1 compile 0m 26s the patch passed with JDK v1.7.0_95
          +1 javac 0m 26s the patch passed
          +1 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 20 unchanged - 1 fixed = 20 total (was 21)
          +1 mvnsite 0m 31s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 15s the patch passed
          +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72
          +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95
          -1 unit 70m 53s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72.
          -1 unit 71m 37s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          158m 55s



          Reason Tests
          JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790128/0007-YARN-4465.patch
          JIRA Issue YARN-4465
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b924458d87ba 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d7fdec1
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10650/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10650/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10650/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10650/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10650/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10650/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 43s trunk passed +1 compile 0m 27s trunk passed with JDK v1.8.0_72 +1 compile 0m 29s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 35s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 4s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 29s the patch passed +1 compile 0m 24s the patch passed with JDK v1.8.0_72 +1 javac 0m 24s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_95 +1 javac 0m 26s the patch passed +1 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 20 unchanged - 1 fixed = 20 total (was 21) +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 15s the patch passed +1 javadoc 0m 18s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95 -1 unit 70m 53s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. -1 unit 71m 37s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 158m 55s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790128/0007-YARN-4465.patch JIRA Issue YARN-4465 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b924458d87ba 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d7fdec1 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/10650/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10650/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10650/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10650/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10650/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/10650/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks Bibin A Chundatt, +1 to latest patch. Will commit tomorrow if there's no objections.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks Bibin A Chundatt , +1 to latest patch. Will commit tomorrow if there's no objections.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Wangda Tan/Sunil G
          Thank you for the reviews.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Wangda Tan / Sunil G Thank you for the reviews.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Committed to trunk/branch-2/branch-2.8.

          Thanks Bibin A Chundatt and thanks reviews from Sunil G!

          Show
          leftnoteasy Wangda Tan added a comment - Committed to trunk/branch-2/branch-2.8. Thanks Bibin A Chundatt and thanks reviews from Sunil G !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9444 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9444/)
          YARN-4465. SchedulerUtils#validateRequest for Label check should happen (wangda: rev 0233d4e0ee9947a95c018b1539310fc0bff6c44e)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerUtils.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9444 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9444/ ) YARN-4465 . SchedulerUtils#validateRequest for Label check should happen (wangda: rev 0233d4e0ee9947a95c018b1539310fc0bff6c44e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerUtils.java
          Hide
          Ying Zhang Ying Zhang added a comment -

          Hi Bibin A Chundatt, Wangda Tan, I was trying this fix on my build, and found that after disabling Node Labels, the RM down immediately after restart completes, due to the exception below which was thrown by the code added in this fix. Have you encountered this? My build is based on 2.7.3. It looks to me that we only need to do this check if it is not in recovery. Or have I missed some other fixes?

          SchedulerUtils.java
            public static void normalizeAndValidateRequest(ResourceRequest resReq,
            ... ...{
              Configuration conf = rmContext.getYarnConfiguration();
              // If Node label is not enabled throw exception
              if (null != conf && !YarnConfiguration.areNodeLabelsEnabled(conf)) {
                String labelExp = resReq.getNodeLabelExpression();
                if (!(RMNodeLabelsManager.NO_LABEL.equals(labelExp)
                    || null == labelExp)) {
                  throw new InvalidLabelResourceRequestException(
                      "Invalid resource request, node label not enabled "
                          + "but request contains label expression");
                }
              }
          

          2016-12-26 23:37:25,844 FATAL resourcemanager.ResourceManager (ResourceManager.java:main(1189)) - Error starting ResourceManager
          org.apache.hadoop.service.ServiceStateException: org.apache.hadoop.yarn.exceptions.InvalidLabelResourceRequestException: Invalid resource request, node label not enabled but request contains label expression
          at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
          at org.apache.hadoop.service.AbstractService.start(AbstractService.java:204)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:972)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1013)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1009)
          at java.security.AccessController.doPrivileged(Native Method)
          at javax.security.auth.Subject.doAs(Subject.java:422)
          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1697)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1009)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1049)
          at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1186)
          Caused by: org.apache.hadoop.yarn.exceptions.InvalidLabelResourceRequestException: Invalid resource request, node label not enabled but request contains label expression
          at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:225)
          at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:248)
          at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.validateAndCreateResourceRequest(RMAppManager.java:394)
          at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:339)
          at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recoverApplication(RMAppManager.java:319)
          at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recover(RMAppManager.java:436)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.recover(ResourceManager.java:1165)
          at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:574)
          at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
          ... 10 more

          Show
          Ying Zhang Ying Zhang added a comment - Hi Bibin A Chundatt , Wangda Tan , I was trying this fix on my build, and found that after disabling Node Labels, the RM down immediately after restart completes, due to the exception below which was thrown by the code added in this fix. Have you encountered this? My build is based on 2.7.3. It looks to me that we only need to do this check if it is not in recovery. Or have I missed some other fixes? SchedulerUtils.java public static void normalizeAndValidateRequest(ResourceRequest resReq, ... ...{ Configuration conf = rmContext.getYarnConfiguration(); // If Node label is not enabled throw exception if ( null != conf && !YarnConfiguration.areNodeLabelsEnabled(conf)) { String labelExp = resReq.getNodeLabelExpression(); if (!(RMNodeLabelsManager.NO_LABEL.equals(labelExp) || null == labelExp)) { throw new InvalidLabelResourceRequestException( "Invalid resource request, node label not enabled " + "but request contains label expression" ); } } 2016-12-26 23:37:25,844 FATAL resourcemanager.ResourceManager (ResourceManager.java:main(1189)) - Error starting ResourceManager org.apache.hadoop.service.ServiceStateException: org.apache.hadoop.yarn.exceptions.InvalidLabelResourceRequestException: Invalid resource request, node label not enabled but request contains label expression at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:204) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:972) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1013) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1009) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1697) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1009) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1049) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1186) Caused by: org.apache.hadoop.yarn.exceptions.InvalidLabelResourceRequestException: Invalid resource request, node label not enabled but request contains label expression at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:225) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:248) at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.validateAndCreateResourceRequest(RMAppManager.java:394) at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:339) at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recoverApplication(RMAppManager.java:319) at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recover(RMAppManager.java:436) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.recover(ResourceManager.java:1165) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:574) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) ... 10 more
          Hide
          sunilg Sunil G added a comment -

          After RM restart, it seems apps are still sending resource requests with NodeLabel Expression in its resource requests.

          Show
          sunilg Sunil G added a comment - After RM restart, it seems apps are still sending resource requests with NodeLabel Expression in its resource requests.
          Hide
          sunilg Sunil G added a comment -

          I guess I missed one call more call flow. After recovering an app after RM restart, AM containers resource request will be validated. if that resource request is invalid due to the presence of node label expression, then this scenario can occur. But that exception is considered as FATAL here. I think we need to retrospect whether we can skip that app or not. I feel we can make that APP failed after recovery. cc/Rohith Sharma K S also.

          Show
          sunilg Sunil G added a comment - I guess I missed one call more call flow. After recovering an app after RM restart, AM containers resource request will be validated. if that resource request is invalid due to the presence of node label expression, then this scenario can occur. But that exception is considered as FATAL here. I think we need to retrospect whether we can skip that app or not. I feel we can make that APP failed after recovery. cc/ Rohith Sharma K S also.
          Hide
          Ying Zhang Ying Zhang added a comment -

          Hi Sunil G, can we just skip the check if it is in recovery, i.e., by adding a "if (!isRecovery)"?

          Show
          Ying Zhang Ying Zhang added a comment - Hi Sunil G , can we just skip the check if it is in recovery, i.e., by adding a "if (!isRecovery)"?
          Hide
          sunilg Sunil G added a comment -

          Recovery of the app has to fail as well. But other apps recovery should go through and RM has to come up. If we are keeping these two as the expected outcome, i think a new jira could be raised and we can continue discussing the same there.

          One more question though? I know its a negative scenario. But node labels were disabled manually before RM restart, correct?

          Show
          sunilg Sunil G added a comment - Recovery of the app has to fail as well. But other apps recovery should go through and RM has to come up. If we are keeping these two as the expected outcome, i think a new jira could be raised and we can continue discussing the same there. One more question though? I know its a negative scenario. But node labels were disabled manually before RM restart, correct?
          Hide
          leftnoteasy Wangda Tan added a comment -

          Nice catch, thanks Ying Zhang!

          I think to solve the problem, we need to swallow the InvalidResourceRequest exception when recovering, and fail the application instead of fail the RM.

          Could you file a ticket for this, we can help with reviews.

          Show
          leftnoteasy Wangda Tan added a comment - Nice catch, thanks Ying Zhang ! I think to solve the problem, we need to swallow the InvalidResourceRequest exception when recovering, and fail the application instead of fail the RM. Could you file a ticket for this, we can help with reviews.
          Hide
          bibinchundatt Bibin A Chundatt added a comment - - edited

          Thank you Ying Zhang
          Apologies for missing out the recover case with enable to disable scenario..
          As Sunil G/Wangda Tan mentioned please do file a jira for the same

          Show
          bibinchundatt Bibin A Chundatt added a comment - - edited Thank you Ying Zhang Apologies for missing out the recover case with enable to disable scenario.. As Sunil G / Wangda Tan mentioned please do file a jira for the same
          Hide
          Ying Zhang Ying Zhang added a comment - - edited

          Thanks Wangda Tan, Sunil G and Bibin A Chundatt. I've created YARN-6031 to track this. Please do comment there, I'll try to come up with a patch soon. And also please feel free to take over

          Show
          Ying Zhang Ying Zhang added a comment - - edited Thanks Wangda Tan , Sunil G and Bibin A Chundatt . I've created YARN-6031 to track this. Please do comment there, I'll try to come up with a patch soon. And also please feel free to take over
          Hide
          Ying Zhang Ying Zhang added a comment -

          Yes, you're right. Please see the repro steps in YARN-6031.

          Show
          Ying Zhang Ying Zhang added a comment - Yes, you're right. Please see the repro steps in YARN-6031 .

            People

            • Assignee:
              bibinchundatt Bibin A Chundatt
              Reporter:
              bibinchundatt Bibin A Chundatt
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development