Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4925

ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently with nodelabel AMRMClient will not be able to specify nodelabels with Node/Rack requests.For application like spark NODE_LOCAL requests cannot be asked with label expression.
      As per the check in AMRMClientImpl#checkNodeLabelExpression

          // Don't allow specify node label against ANY request
          if ((containerRequest.getRacks() != null && 
              (!containerRequest.getRacks().isEmpty()))
              || 
              (containerRequest.getNodes() != null && 
              (!containerRequest.getNodes().isEmpty()))) {
            throw new InvalidContainerRequestException(
                "Cannot specify node label with rack and node");
          }
      

      AppSchedulingInfo#updateResourceRequests we do reset of labels to that of OFF-SWITCH.

      The above check is not required for ContainerRequest ask /cc Tan, Wangda thank you for confirming

      1. 0001-YARN-4925.patch
        2 kB
        Bibin A Chundatt
      2. 0002-YARN-4925.patch
        3 kB
        Bibin A Chundatt
      3. YARN-4925-branch-2.7.001.patch
        3 kB
        Jonathan Hung

        Issue Links

          Activity

          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Bibin A Chundatt,
          At first glance seems to be a valid issue, IIUC we can directly apply the nodelabel expression to the any request and no need to have this check.

          Show
          Naganarasimha Naganarasimha G R added a comment - Bibin A Chundatt , At first glance seems to be a valid issue, IIUC we can directly apply the nodelabel expression to the any request and no need to have this check.
          Hide
          sunilg Sunil G added a comment -

          Hi Bibin A Chundatt
          I have one doubt here. After removing this check, now ResourceRequest can have label exp for NodeLocal or RackLocal or ANY. With YARN-4140, label specified in ANY will reset all other ResourceRequest's label expression (for same priority). Is this intended OR will it solve the case mentioned in Spark scenario?

          Show
          sunilg Sunil G added a comment - Hi Bibin A Chundatt I have one doubt here. After removing this check, now ResourceRequest can have label exp for NodeLocal or RackLocal or ANY. With YARN-4140 , label specified in ANY will reset all other ResourceRequest's label expression (for same priority). Is this intended OR will it solve the case mentioned in Spark scenario?
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          IIUC, Bibin A Chundatt's solution was to keep the code same i.e. set the nodelabel expression present containerRequest for only any request thus user will be able to specify node/rack locality with label expression.

          Show
          Naganarasimha Naganarasimha G R added a comment - IIUC, Bibin A Chundatt 's solution was to keep the code same i.e. set the nodelabel expression present containerRequest for only any request thus user will be able to specify node/rack locality with label expression.
          Hide
          sunilg Sunil G added a comment -

          Thanks NGarla_Unused for the clarification. One final doubt, in this case if we change the labelExpression for ANY containerRequest , it will replace the expression set by node/rack local earlier. Correct.

          Show
          sunilg Sunil G added a comment - Thanks NGarla_Unused for the clarification. One final doubt, in this case if we change the labelExpression for ANY containerRequest , it will replace the expression set by node/rack local earlier. Correct.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Sunil G and Naganarasimha G R
          Thank you for looking into the issue.

          The check is for ContainerRequest and IIUC after the discussion with Tan, Wangda the check was done to properly account how much resources an app requests for each partitions considering that label for NODE_LOCAL and OFF-SWITCH shouldnt be set differently, which is not possible if we are setting ContainerRequest .

           public static class ContainerRequest {
              final Resource capability;
              final List<String> nodes;
              final List<String> racks;
              final Priority priority;
              final boolean relaxLocality;
              final String nodeLabelsExpression;
          }
          

          Consider in one NODELABEL there are 100 Nodes and need container to be started on NODE where data is available this is not allowed.For issue we face was for spark container needed be node local and was not allowed.
          cc/ Tan, Wangda

          Show
          bibinchundatt Bibin A Chundatt added a comment - Sunil G and Naganarasimha G R Thank you for looking into the issue. The check is for ContainerRequest and IIUC after the discussion with Tan, Wangda the check was done to properly account how much resources an app requests for each partitions considering that label for NODE_LOCAL and OFF-SWITCH shouldnt be set differently, which is not possible if we are setting ContainerRequest . public static class ContainerRequest { final Resource capability; final List<String> nodes; final List<String> racks; final Priority priority; final boolean relaxLocality; final String nodeLabelsExpression; } Consider in one NODELABEL there are 100 Nodes and need container to be started on NODE where data is available this is not allowed.For issue we face was for spark container needed be node local and was not allowed. cc/ Tan, Wangda
          Hide
          sunilg Sunil G added a comment -

          Thanks Bibin A Chundatt.
          Yes, NODE_LOCAL and OFF-SWITCH should not be set differently. So a change in ANY can reset other 2 types which is fine. I was just ensuring this point. No issue

          Show
          sunilg Sunil G added a comment - Thanks Bibin A Chundatt . Yes, NODE_LOCAL and OFF-SWITCH should not be set differently. So a change in ANY can reset other 2 types which is fine. I was just ensuring this point. No issue
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Attaching patch for the same

          Show
          bibinchundatt Bibin A Chundatt added a comment - Attaching patch for the same
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 46s trunk passed
          +1 compile 0m 17s trunk passed with JDK v1.8.0_77
          +1 compile 0m 18s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 15s trunk passed
          +1 mvnsite 0m 21s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 0m 34s trunk passed
          +1 javadoc 0m 14s trunk passed with JDK v1.8.0_77
          +1 javadoc 0m 18s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 18s the patch passed
          +1 compile 0m 13s the patch passed with JDK v1.8.0_77
          +1 javac 0m 13s the patch passed
          +1 compile 0m 16s the patch passed with JDK v1.7.0_95
          +1 javac 0m 16s the patch passed
          +1 checkstyle 0m 12s the patch passed
          +1 mvnsite 0m 19s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 0m 42s the patch passed
          +1 javadoc 0m 12s the patch passed with JDK v1.8.0_77
          +1 javadoc 0m 15s the patch passed with JDK v1.7.0_95
          -1 unit 66m 11s hadoop-yarn-client in the patch failed with JDK v1.8.0_77.
          -1 unit 66m 23s hadoop-yarn-client in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 18s Patch does not generate ASF License warnings.
          146m 0s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.yarn.client.api.impl.TestAMRMProxy
            hadoop.yarn.client.TestGetGroups
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.client.api.impl.TestAMRMProxy
            hadoop.yarn.client.TestGetGroups
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800420/0001-YARN-4925.patch
          JIRA Issue YARN-4925
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 726f1e02ecc1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6fcde2e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11196/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11196/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11196/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11196/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11196/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11196/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 46s trunk passed +1 compile 0m 17s trunk passed with JDK v1.8.0_77 +1 compile 0m 18s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 34s trunk passed +1 javadoc 0m 14s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 18s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 18s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_77 +1 javac 0m 13s the patch passed +1 compile 0m 16s the patch passed with JDK v1.7.0_95 +1 javac 0m 16s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 42s the patch passed +1 javadoc 0m 12s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 15s the patch passed with JDK v1.7.0_95 -1 unit 66m 11s hadoop-yarn-client in the patch failed with JDK v1.8.0_77. -1 unit 66m 23s hadoop-yarn-client in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 146m 0s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.client.api.impl.TestAMRMProxy   hadoop.yarn.client.TestGetGroups JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient JDK v1.7.0_95 Failed junit tests hadoop.yarn.client.api.impl.TestAMRMProxy   hadoop.yarn.client.TestGetGroups JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800420/0001-YARN-4925.patch JIRA Issue YARN-4925 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 726f1e02ecc1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6fcde2e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11196/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11196/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11196/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11196/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11196/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/11196/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks for the patch Bibin A Chundatt, seems like test cases are mostly not related to the patch please cross check once and hope a test case can also be added.

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks for the patch Bibin A Chundatt , seems like test cases are mostly not related to the patch please cross check once and hope a test case can also be added.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Attaching patch after adding testcase

          Show
          bibinchundatt Bibin A Chundatt added a comment - Attaching patch after adding testcase
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 39s trunk passed
          +1 compile 0m 15s trunk passed with JDK v1.8.0_92
          +1 compile 0m 19s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 14s trunk passed
          +1 mvnsite 0m 22s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 34s trunk passed
          +1 javadoc 0m 14s trunk passed with JDK v1.8.0_92
          +1 javadoc 0m 17s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 18s the patch passed
          +1 compile 0m 12s the patch passed with JDK v1.8.0_92
          +1 javac 0m 12s the patch passed
          +1 compile 0m 16s the patch passed with JDK v1.7.0_95
          +1 javac 0m 16s the patch passed
          +1 checkstyle 0m 12s the patch passed
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 0m 42s the patch passed
          +1 javadoc 0m 11s the patch passed with JDK v1.8.0_92
          +1 javadoc 0m 15s the patch passed with JDK v1.7.0_95
          -1 unit 66m 1s hadoop-yarn-client in the patch failed with JDK v1.8.0_92.
          -1 unit 66m 15s hadoop-yarn-client in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          145m 34s



          Reason Tests
          JDK v1.8.0_92 Failed junit tests hadoop.yarn.client.TestGetGroups
            hadoop.yarn.client.api.impl.TestAMRMProxy
          JDK v1.8.0_92 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.client.TestGetGroups
            hadoop.yarn.client.api.impl.TestAMRMProxy
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801588/0002-YARN-4925.patch
          JIRA Issue YARN-4925
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux a4775a05a51f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 2753185
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11292/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_92.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11292/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11292/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_92.txt https://builds.apache.org/job/PreCommit-YARN-Build/11292/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11292/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11292/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 39s trunk passed +1 compile 0m 15s trunk passed with JDK v1.8.0_92 +1 compile 0m 19s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 34s trunk passed +1 javadoc 0m 14s trunk passed with JDK v1.8.0_92 +1 javadoc 0m 17s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 18s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_92 +1 javac 0m 12s the patch passed +1 compile 0m 16s the patch passed with JDK v1.7.0_95 +1 javac 0m 16s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 42s the patch passed +1 javadoc 0m 11s the patch passed with JDK v1.8.0_92 +1 javadoc 0m 15s the patch passed with JDK v1.7.0_95 -1 unit 66m 1s hadoop-yarn-client in the patch failed with JDK v1.8.0_92. -1 unit 66m 15s hadoop-yarn-client in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 145m 34s Reason Tests JDK v1.8.0_92 Failed junit tests hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.api.impl.TestAMRMProxy JDK v1.8.0_92 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient JDK v1.7.0_95 Failed junit tests hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.api.impl.TestAMRMProxy JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801588/0002-YARN-4925.patch JIRA Issue YARN-4925 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a4775a05a51f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2753185 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11292/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_92.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11292/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11292/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_92.txt https://builds.apache.org/job/PreCommit-YARN-Build/11292/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11292/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/11292/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Naganarasimha G R/Wangda Tan
          Could you please review patch attached.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Naganarasimha G R / Wangda Tan Could you please review patch attached.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          +1 Patch LG

          Show
          Naganarasimha Naganarasimha G R added a comment - +1 Patch LG
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Will commit it, if no one has any objections...

          Show
          Naganarasimha Naganarasimha G R added a comment - Will commit it, if no one has any objections...
          Hide
          Naganarasimha Naganarasimha G R added a comment - - edited

          Committed it to branch 2.8, branch 2 & trunk, Thanks for working on this patch Bibin A Chundatt and Sunil G for the review.

          Show
          Naganarasimha Naganarasimha G R added a comment - - edited Committed it to branch 2.8, branch 2 & trunk, Thanks for working on this patch Bibin A Chundatt and Sunil G for the review.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9816 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9816/)
          YARN-4925. ContainerRequest in AMRMClient, application should be able to (naganarasimha_gr: rev f04c81c9ce93512bc714531a7731debbe6b794ce)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMClient.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AMRMClientImpl.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9816 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9816/ ) YARN-4925 . ContainerRequest in AMRMClient, application should be able to (naganarasimha_gr: rev f04c81c9ce93512bc714531a7731debbe6b794ce) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMClient.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AMRMClientImpl.java
          Hide
          jhung Jonathan Hung added a comment -

          Attached a branch-2.7 patch.

          Show
          jhung Jonathan Hung added a comment - Attached a branch-2.7 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 5m 42s branch-2.7 passed
          +1 compile 0m 13s branch-2.7 passed with JDK v1.8.0_131
          +1 compile 0m 16s branch-2.7 passed with JDK v1.7.0_131
          +1 checkstyle 0m 12s branch-2.7 passed
          +1 mvnsite 0m 20s branch-2.7 passed
          +1 mvneclipse 0m 10s branch-2.7 passed
          +1 findbugs 0m 29s branch-2.7 passed
          +1 javadoc 0m 11s branch-2.7 passed with JDK v1.8.0_131
          +1 javadoc 0m 14s branch-2.7 passed with JDK v1.7.0_131
          +1 mvninstall 0m 15s the patch passed
          +1 compile 0m 10s the patch passed with JDK v1.8.0_131
          +1 javac 0m 10s the patch passed
          +1 compile 0m 15s the patch passed with JDK v1.7.0_131
          +1 javac 0m 15s the patch passed
          +1 checkstyle 0m 10s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 0 new + 215 unchanged - 6 fixed = 215 total (was 221)
          +1 mvnsite 0m 18s the patch passed
          +1 mvneclipse 0m 9s the patch passed
          -1 whitespace 0m 0s The patch has 1999 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 0m 29s The patch 75 line(s) with tabs.
          +1 findbugs 0m 37s the patch passed
          +1 javadoc 0m 8s the patch passed with JDK v1.8.0_131
          +1 javadoc 0m 11s the patch passed with JDK v1.7.0_131
          -1 unit 46m 39s hadoop-yarn-client in the patch failed with JDK v1.7.0_131.
          +1 asflicense 0m 14s The patch does not generate ASF License warnings.
          120m 6s



          Reason Tests
          JDK v1.8.0_131 Failed junit tests hadoop.yarn.client.TestGetGroups
            hadoop.yarn.client.TestResourceTrackerOnHA
            hadoop.yarn.client.TestApplicationClientProtocolOnHA
            hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA
          JDK v1.8.0_131 Timed out junit tests org.apache.hadoop.yarn.client.TestRMFailover
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient
          JDK v1.7.0_131 Failed junit tests hadoop.yarn.client.TestGetGroups
            hadoop.yarn.client.TestResourceTrackerOnHA
            hadoop.yarn.client.TestApplicationClientProtocolOnHA
            hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA
            hadoop.yarn.client.TestRMFailover
          JDK v1.7.0_131 Timed out junit tests org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:c420dfe
          JIRA Issue YARN-4925
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869043/YARN-4925-branch-2.7.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c67061db23c7 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / eecc68d
          Default Java 1.7.0_131
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15982/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15982/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/15982/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_131.txt
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15982/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/15982/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 5m 42s branch-2.7 passed +1 compile 0m 13s branch-2.7 passed with JDK v1.8.0_131 +1 compile 0m 16s branch-2.7 passed with JDK v1.7.0_131 +1 checkstyle 0m 12s branch-2.7 passed +1 mvnsite 0m 20s branch-2.7 passed +1 mvneclipse 0m 10s branch-2.7 passed +1 findbugs 0m 29s branch-2.7 passed +1 javadoc 0m 11s branch-2.7 passed with JDK v1.8.0_131 +1 javadoc 0m 14s branch-2.7 passed with JDK v1.7.0_131 +1 mvninstall 0m 15s the patch passed +1 compile 0m 10s the patch passed with JDK v1.8.0_131 +1 javac 0m 10s the patch passed +1 compile 0m 15s the patch passed with JDK v1.7.0_131 +1 javac 0m 15s the patch passed +1 checkstyle 0m 10s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 0 new + 215 unchanged - 6 fixed = 215 total (was 221) +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 9s the patch passed -1 whitespace 0m 0s The patch has 1999 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 0m 29s The patch 75 line(s) with tabs. +1 findbugs 0m 37s the patch passed +1 javadoc 0m 8s the patch passed with JDK v1.8.0_131 +1 javadoc 0m 11s the patch passed with JDK v1.7.0_131 -1 unit 46m 39s hadoop-yarn-client in the patch failed with JDK v1.7.0_131. +1 asflicense 0m 14s The patch does not generate ASF License warnings. 120m 6s Reason Tests JDK v1.8.0_131 Failed junit tests hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.TestResourceTrackerOnHA   hadoop.yarn.client.TestApplicationClientProtocolOnHA   hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA JDK v1.8.0_131 Timed out junit tests org.apache.hadoop.yarn.client.TestRMFailover   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient JDK v1.7.0_131 Failed junit tests hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.TestResourceTrackerOnHA   hadoop.yarn.client.TestApplicationClientProtocolOnHA   hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA   hadoop.yarn.client.TestRMFailover JDK v1.7.0_131 Timed out junit tests org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue YARN-4925 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869043/YARN-4925-branch-2.7.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c67061db23c7 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / eecc68d Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15982/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15982/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15982/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_131.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15982/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/15982/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 16m 3s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 18s branch-2.7 passed
          +1 compile 0m 17s branch-2.7 passed with JDK v1.8.0_131
          +1 compile 0m 21s branch-2.7 passed with JDK v1.7.0_131
          +1 checkstyle 0m 17s branch-2.7 passed
          +1 mvnsite 0m 26s branch-2.7 passed
          +1 mvneclipse 0m 14s branch-2.7 passed
          +1 findbugs 0m 34s branch-2.7 passed
          +1 javadoc 0m 13s branch-2.7 passed with JDK v1.8.0_131
          +1 javadoc 0m 15s branch-2.7 passed with JDK v1.7.0_131
          +1 mvninstall 0m 17s the patch passed
          +1 compile 0m 13s the patch passed with JDK v1.8.0_131
          +1 javac 0m 13s the patch passed
          +1 compile 0m 15s the patch passed with JDK v1.7.0_131
          +1 javac 0m 15s the patch passed
          +1 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 0 new + 215 unchanged - 6 fixed = 215 total (was 221)
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          -1 whitespace 0m 0s The patch has 2346 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 0m 55s The patch 75 line(s) with tabs.
          +1 findbugs 0m 43s the patch passed
          +1 javadoc 0m 11s the patch passed with JDK v1.8.0_131
          +1 javadoc 0m 14s the patch passed with JDK v1.7.0_131
          -1 unit 46m 41s hadoop-yarn-client in the patch failed with JDK v1.7.0_131.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          142m 36s



          Reason Tests
          JDK v1.8.0_131 Failed junit tests hadoop.yarn.client.TestApplicationClientProtocolOnHA
            hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA
            hadoop.yarn.client.TestGetGroups
            hadoop.yarn.client.TestResourceTrackerOnHA
          JDK v1.8.0_131 Timed out junit tests org.apache.hadoop.yarn.client.TestRMFailover
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient
          JDK v1.7.0_131 Failed junit tests hadoop.yarn.client.TestApplicationClientProtocolOnHA
            hadoop.yarn.client.TestRMFailover
            hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA
            hadoop.yarn.client.TestGetGroups
            hadoop.yarn.client.TestResourceTrackerOnHA
          JDK v1.7.0_131 Timed out junit tests org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:c420dfe
          JIRA Issue YARN-4925
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869043/YARN-4925-branch-2.7.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 875d4c2c4d38 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / eecc68d
          Default Java 1.7.0_131
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15985/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15985/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/15985/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_131.txt
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15985/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/15985/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 16m 3s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 18s branch-2.7 passed +1 compile 0m 17s branch-2.7 passed with JDK v1.8.0_131 +1 compile 0m 21s branch-2.7 passed with JDK v1.7.0_131 +1 checkstyle 0m 17s branch-2.7 passed +1 mvnsite 0m 26s branch-2.7 passed +1 mvneclipse 0m 14s branch-2.7 passed +1 findbugs 0m 34s branch-2.7 passed +1 javadoc 0m 13s branch-2.7 passed with JDK v1.8.0_131 +1 javadoc 0m 15s branch-2.7 passed with JDK v1.7.0_131 +1 mvninstall 0m 17s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_131 +1 javac 0m 13s the patch passed +1 compile 0m 15s the patch passed with JDK v1.7.0_131 +1 javac 0m 15s the patch passed +1 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 0 new + 215 unchanged - 6 fixed = 215 total (was 221) +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 11s the patch passed -1 whitespace 0m 0s The patch has 2346 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 0m 55s The patch 75 line(s) with tabs. +1 findbugs 0m 43s the patch passed +1 javadoc 0m 11s the patch passed with JDK v1.8.0_131 +1 javadoc 0m 14s the patch passed with JDK v1.7.0_131 -1 unit 46m 41s hadoop-yarn-client in the patch failed with JDK v1.7.0_131. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 142m 36s Reason Tests JDK v1.8.0_131 Failed junit tests hadoop.yarn.client.TestApplicationClientProtocolOnHA   hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA   hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.TestResourceTrackerOnHA JDK v1.8.0_131 Timed out junit tests org.apache.hadoop.yarn.client.TestRMFailover   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient JDK v1.7.0_131 Failed junit tests hadoop.yarn.client.TestApplicationClientProtocolOnHA   hadoop.yarn.client.TestRMFailover   hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA   hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.TestResourceTrackerOnHA JDK v1.7.0_131 Timed out junit tests org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue YARN-4925 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869043/YARN-4925-branch-2.7.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 875d4c2c4d38 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / eecc68d Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15985/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/15985/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15985/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_131.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15985/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/15985/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          shv Konstantin Shvachko added a comment - - edited

          Hey Jonathan Hung looks like there are problems with a few tests, including TestAMRMClient on branch-2.7.
          Could you please check.

          Show
          shv Konstantin Shvachko added a comment - - edited Hey Jonathan Hung looks like there are problems with a few tests, including TestAMRMClient on branch-2.7. Could you please check.
          Hide
          jhung Jonathan Hung added a comment -

          I'm not able to reproduce some of the test failures:

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.yarn.client.TestApplicationClientProtocolOnHA
          Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 79.298 sec - in org.apache.hadoop.yarn.client.TestApplicationClientProtocolOnHA
          Running org.apache.hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA
          Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.953 sec - in org.apache.hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA
          Running org.apache.hadoop.yarn.client.TestGetGroups
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.963 sec - in org.apache.hadoop.yarn.client.TestGetGroups
          Running org.apache.hadoop.yarn.client.TestResourceTrackerOnHA
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.216 sec - in org.apache.hadoop.yarn.client.TestResourceTrackerOnHA
          Running org.apache.hadoop.yarn.client.TestRMFailover
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.319 sec - in org.apache.hadoop.yarn.client.TestRMFailover
          
          Results :
          
          Tests run: 32, Failures: 0, Errors: 0, Skipped: 0
          

          Seems to be related to the jenkins environment.

          Will look into TestAMRMClient, TestYarnClient, TestNMClient

          Show
          jhung Jonathan Hung added a comment - I'm not able to reproduce some of the test failures: ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.yarn.client.TestApplicationClientProtocolOnHA Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 79.298 sec - in org.apache.hadoop.yarn.client.TestApplicationClientProtocolOnHA Running org.apache.hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.953 sec - in org.apache.hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA Running org.apache.hadoop.yarn.client.TestGetGroups Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.963 sec - in org.apache.hadoop.yarn.client.TestGetGroups Running org.apache.hadoop.yarn.client.TestResourceTrackerOnHA Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.216 sec - in org.apache.hadoop.yarn.client.TestResourceTrackerOnHA Running org.apache.hadoop.yarn.client.TestRMFailover Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.319 sec - in org.apache.hadoop.yarn.client.TestRMFailover Results : Tests run: 32, Failures: 0, Errors: 0, Skipped: 0 Seems to be related to the jenkins environment. Will look into TestAMRMClient, TestYarnClient, TestNMClient
          Hide
          bibinchundatt Bibin A Chundatt added a comment - - edited

          We should also backport dependent YARN-4140

          Show
          bibinchundatt Bibin A Chundatt added a comment - - edited We should also backport dependent YARN-4140
          Hide
          jhung Jonathan Hung added a comment -

          Thanks Bibin A Chundatt I am planning on working on this as well.

          Seems the test failures are possibly related to YARN-5208, which depends on HADOOP-12954. Created YARN-6662 and HADOOP-14463 for these issues, respectively.

          Show
          jhung Jonathan Hung added a comment - Thanks Bibin A Chundatt I am planning on working on this as well. Seems the test failures are possibly related to YARN-5208 , which depends on HADOOP-12954 . Created YARN-6662 and HADOOP-14463 for these issues, respectively.
          Hide
          jhung Jonathan Hung added a comment -

          Konstantin Shvachko the branch-2.7 patch passes locally for me:

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          
          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
          Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.551 sec - in org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
          
          Results :
          
          Tests run: 10, Failures: 0, Errors: 0, Skipped: 0
          Show
          jhung Jonathan Hung added a comment - Konstantin Shvachko the branch-2.7 patch passes locally for me: ------------------------------------------------------- T E S T S ------------------------------------------------------- ------------------------------------------------------- T E S T S ------------------------------------------------------- Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.yarn.client.api.impl.TestAMRMClient Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.551 sec - in org.apache.hadoop.yarn.client.api.impl.TestAMRMClient Results : Tests run: 10, Failures: 0, Errors: 0, Skipped: 0
          Hide
          shv Konstantin Shvachko added a comment -

          I've done quite a few runs for the client tests.
          I see TestAMRMClient and other client tests failing consistently on my Linux box. On Mac they succeed most of the time, but also fail once in a while. I don't think it is related to this particular patch, but I don't know for sure.
          So I think we should get some understanding on why the tests are failing, which will give us an idea if we can commit this jira and focus on fixing the tests for branch-2.7.

          Show
          shv Konstantin Shvachko added a comment - I've done quite a few runs for the client tests. I see TestAMRMClient and other client tests failing consistently on my Linux box. On Mac they succeed most of the time, but also fail once in a while. I don't think it is related to this particular patch, but I don't know for sure. So I think we should get some understanding on why the tests are failing, which will give us an idea if we can commit this jira and focus on fixing the tests for branch-2.7.
          Hide
          jhung Jonathan Hung added a comment -

          Created YARN-6684 for this issue.
          Seems the unit test failures are unrelated to this patch.

          Show
          jhung Jonathan Hung added a comment - Created YARN-6684 for this issue. Seems the unit test failures are unrelated to this patch.
          Hide
          shv Konstantin Shvachko added a comment -

          Turned out the *Client tests were failing due to unusual local URI configuration, which should be addressed in YARN-6684. I was able to successfully run the tests locally with the work around.

          Show
          shv Konstantin Shvachko added a comment - Turned out the *Client tests were failing due to unusual local URI configuration, which should be addressed in YARN-6684 . I was able to successfully run the tests locally with the work around.
          Hide
          shv Konstantin Shvachko added a comment -

          Just committed this to branch-2.7. Thank you Jonathan Hung for the backport.

          Show
          shv Konstantin Shvachko added a comment - Just committed this to branch-2.7. Thank you Jonathan Hung for the backport.
          Hide
          jhung Jonathan Hung added a comment -
          Show
          jhung Jonathan Hung added a comment - Thanks Konstantin Shvachko !

            People

            • Assignee:
              bibinchundatt Bibin A Chundatt
              Reporter:
              bibinchundatt Bibin A Chundatt
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development