Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5044

Have AM trigger jstack on task attempts that timeout before killing them

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: mr-am
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When an AM expires a task attempt it would be nice if it triggered a jstack output via SIGQUIT before killing the task attempt. This would be invaluable for helping users debug their hung tasks, especially if they do not have shell access to the nodes.

      1. MAPREDUCE-5044.008.patch
        35 kB
        Eric Payne
      2. MAPREDUCE-5044.009.patch
        35 kB
        Eric Payne
      3. MAPREDUCE-5044.010.patch
        42 kB
        Eric Payne
      4. MAPREDUCE-5044.011.patch
        58 kB
        Eric Payne
      5. MAPREDUCE-5044.012.patch
        60 kB
        Eric Payne
      6. MAPREDUCE-5044.013.patch
        60 kB
        Eric Payne
      7. MAPREDUCE-5044.v01.patch
        67 kB
        Gera Shegalov
      8. MAPREDUCE-5044.v02.patch
        11 kB
        Gera Shegalov
      9. MAPREDUCE-5044.v03.patch
        6 kB
        Gera Shegalov
      10. MAPREDUCE-5044.v04.patch
        12 kB
        Gera Shegalov
      11. MAPREDUCE-5044.v05.patch
        15 kB
        Gera Shegalov
      12. MAPREDUCE-5044.v06.patch
        16 kB
        Gera Shegalov
      13. MAPREDUCE-5044.v07.local.patch
        35 kB
        Eric Payne
      14. Screen Shot 2013-11-12 at 1.05.32 PM.png
        40 kB
        Gera Shegalov
      15. Screen Shot 2013-11-12 at 1.06.04 PM.png
        191 kB
        Gera Shegalov

        Issue Links

          Activity

          Hide
          jlowe Jason Lowe added a comment -

          This is basically a reimplementation of MAPREDUCE-1119 for the MapReduce ApplicationMaster.

          Show
          jlowe Jason Lowe added a comment - This is basically a reimplementation of MAPREDUCE-1119 for the MapReduce ApplicationMaster.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Test with
          $ bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-*-tests.jar sleep -Dmapreduce.task.timeout=$((30*1000)) -m 1 -mt $((2*60*1000)) -r 0 -recordt $((2*60*1000))

          Show
          jira.shegalov Gera Shegalov added a comment - Test with $ bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-*-tests.jar sleep -Dmapreduce.task.timeout=$((30*1000)) -m 1 -mt $((2*60*1000)) -r 0 -recordt $((2*60*1000))
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12614676/Screen%20Shot%202013-11-12%20at%201.06.04%20PM.png
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4215//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12614676/Screen%20Shot%202013-11-12%20at%201.06.04%20PM.png against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4215//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12614688/MAPREDUCE-5044.v01.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4216//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4216//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12614688/MAPREDUCE-5044.v01.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 9 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4216//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4216//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12614748/Screen%20Shot%202013-11-12%20at%201.06.04%20PM.png
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4218//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12614748/Screen%20Shot%202013-11-12%20at%201.06.04%20PM.png against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4218//console This message is automatically generated.
          Hide
          sandyr Sandy Ryza added a comment -

          This should depend on YARN-445, right?

          Show
          sandyr Sandy Ryza added a comment - This should depend on YARN-445 , right?
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Our patch does not depend on YARN-445. In the specific scenario of a task timeout there is no need for an extra RPC.

          Show
          jira.shegalov Gera Shegalov added a comment - Our patch does not depend on YARN-445 . In the specific scenario of a task timeout there is no need for an extra RPC.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          I do think this is tied to YARN-445. But we can discuss that elsewhere.

          This patch has YARN changes. YARN and MapReduce are split into separate sub-modules of Hadoop. Please file a YARN ticket for the changes you need in YARN here: https://issues.apache.org/jira/browse/YARN

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - I do think this is tied to YARN-445 . But we can discuss that elsewhere. This patch has YARN changes. YARN and MapReduce are split into separate sub-modules of Hadoop. Please file a YARN ticket for the changes you need in YARN here: https://issues.apache.org/jira/browse/YARN
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Moved YARN-related changes into YARN-1515

          Show
          jira.shegalov Gera Shegalov added a comment - Moved YARN-related changes into YARN-1515
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Another attempt to remove yarn-related changes.

          Show
          jira.shegalov Gera Shegalov added a comment - Another attempt to remove yarn-related changes.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Hi Vinod Kumar Vavilapalli, thanks for chiming in. Please review MAPREDUCE-5044 and YARN-1515.

          Show
          jira.shegalov Gera Shegalov added a comment - Hi Vinod Kumar Vavilapalli , thanks for chiming in. Please review MAPREDUCE-5044 and YARN-1515 .
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12619497/MAPREDUCE-5044.v03.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4269//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12619497/MAPREDUCE-5044.v03.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4269//console This message is automatically generated.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          v04 to apply on top of YARN-1515.v05. It now makes sure that a thread dump is created in the uber mode.

          Added unit tests for a normal MR job and uber MR job.

          While working on this I realized that we actually need to discuss how mapreduce.task.timeout is treated in the ubermode. Right now it's basically ignored because AM does not kill itself, LocalContainerLauncher processes CONTAINER_REMOTE_CLEANUP inline with the stuck in SubtaskRunner. The liveness monitor for AM in RM does not catch the problem either because RMCommunicator heartbeats in a separate allocator thread.

          I am considering two options:

          • move heartbeat() into SubtaskRunner for ubermode such that the liveness monitor catches the stuck ubertask.
          • do System.exit(errorcode) when TA_TIMEOUT occurs.
          Show
          jira.shegalov Gera Shegalov added a comment - v04 to apply on top of YARN-1515 .v05. It now makes sure that a thread dump is created in the uber mode. Added unit tests for a normal MR job and uber MR job. While working on this I realized that we actually need to discuss how mapreduce.task.timeout is treated in the ubermode. Right now it's basically ignored because AM does not kill itself, LocalContainerLauncher processes CONTAINER_REMOTE_CLEANUP inline with the stuck in SubtaskRunner. The liveness monitor for AM in RM does not catch the problem either because RMCommunicator heartbeats in a separate allocator thread. I am considering two options: move heartbeat() into SubtaskRunner for ubermode such that the liveness monitor catches the stuck ubertask. do System.exit(errorcode) when TA_TIMEOUT occurs.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12630788/MAPREDUCE-5044.v04.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4367//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12630788/MAPREDUCE-5044.v04.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4367//console This message is automatically generated.
          Hide
          mingma Ming Ma added a comment -

          This is quite useful. Can we get this and YARN-1515 in 2.4.0 release?

          Show
          mingma Ming Ma added a comment - This is quite useful. Can we get this and YARN-1515 in 2.4.0 release?
          Hide
          jira.shegalov Gera Shegalov added a comment -

          v05 rebase and the AM thread dump coverage by unit test.

          Show
          jira.shegalov Gera Shegalov added a comment - v05 rebase and the AM thread dump coverage by unit test.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12644057/MAPREDUCE-5044.v05.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4593//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12644057/MAPREDUCE-5044.v05.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4593//console This message is automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          Patch looks good, I just have a minor comment. Rather than add dumpThreads to ContainerLauncherEvent there should be a ContainerRemoteCleanupEvent to hold the fields specific to the cleanup event, just like there's a ContainerRemoteLaunchEvent to hold the fields specific to the remote launch event.

          Show
          jlowe Jason Lowe added a comment - Patch looks good, I just have a minor comment. Rather than add dumpThreads to ContainerLauncherEvent there should be a ContainerRemoteCleanupEvent to hold the fields specific to the cleanup event, just like there's a ContainerRemoteLaunchEvent to hold the fields specific to the remote launch event.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Thanks for reviewing, Jason. ContainerRemoteCleanupEvent remark belongs to YARN-1515 as well, and is addressed there.

          Show
          jira.shegalov Gera Shegalov added a comment - Thanks for reviewing, Jason. ContainerRemoteCleanupEvent remark belongs to YARN-1515 as well, and is addressed there.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          v06 that utilizes signalContainer API provided by YARN-1515.v08.patch

          Show
          jira.shegalov Gera Shegalov added a comment - v06 that utilizes signalContainer API provided by YARN-1515 .v08.patch
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12645521/MAPREDUCE-5044.v06.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4706//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12645521/MAPREDUCE-5044.v06.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4706//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12645521/MAPREDUCE-5044.v06.patch
          against trunk revision c4980a2.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5164//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12645521/MAPREDUCE-5044.v06.patch against trunk revision c4980a2. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5164//console This message is automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          Cancelling patch as it no longer applies.

          Show
          aw Allen Wittenauer added a comment - Cancelling patch as it no longer applies.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Moving features/enhancements out of previously closed releases into the next minor release 2.8.0.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Moving features/enhancements out of previously closed releases into the next minor release 2.8.0.
          Hide
          eepayne Eric Payne added a comment -

          Hi Gera Shegalov. I would like to see this functionality implemented. We occasionally see containers time out, and it would be good if users could have direct feedback in the form of a jstack to help them debug their applications.

          I have been coming up to speed on the work that's already been committed in this area under YARN-445 and its children. IIUC, YARN-445 and its children put in place the infrastructure for a Client -> RM -> NM -> Container signal path. On the other hand, this JIRA (along with YARN-1515) implements an AM -> NM -> Container signal path and the ability to send multiple signals per call.

          It seems that these pieces could possibly be split into separate JIRAs. Either way, I think that a lot of what has been done in this JIRA could be used to add the interface to ContainerManagementProtocol that would allow the AM to prompt the NM to signal the container to dump its stack prior to killing the container on a timeout.

          Is there a possibility that this JIRA will move forward? Ideally, we would like it all ported back to 2.7. Please let me know if there's anything I can do.

          Show
          eepayne Eric Payne added a comment - Hi Gera Shegalov . I would like to see this functionality implemented. We occasionally see containers time out, and it would be good if users could have direct feedback in the form of a jstack to help them debug their applications. I have been coming up to speed on the work that's already been committed in this area under YARN-445 and its children. IIUC, YARN-445 and its children put in place the infrastructure for a Client -> RM -> NM -> Container signal path. On the other hand, this JIRA (along with YARN-1515 ) implements an AM -> NM -> Container signal path and the ability to send multiple signals per call. It seems that these pieces could possibly be split into separate JIRAs. Either way, I think that a lot of what has been done in this JIRA could be used to add the interface to ContainerManagementProtocol that would allow the AM to prompt the NM to signal the container to dump its stack prior to killing the container on a timeout. Is there a possibility that this JIRA will move forward? Ideally, we would like it all ported back to 2.7. Please let me know if there's anything I can do.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Hi Eric Payne, I am glad that we are finally picking it up. These thread dumps along with xprof has made debugging at Twitter so easy. Often you see what's wrong even without looking at the user code. Unfortunately, organizationally, Hadoop is not my current focus (it's been 2+ years since I posted my patch) . I am sure somebody from @TwitterHadoop will help move it along.

          Show
          jira.shegalov Gera Shegalov added a comment - Hi Eric Payne , I am glad that we are finally picking it up. These thread dumps along with xprof has made debugging at Twitter so easy. Often you see what's wrong even without looking at the user code. Unfortunately, organizationally, Hadoop is not my current focus (it's been 2+ years since I posted my patch) . I am sure somebody from @TwitterHadoop will help move it along.
          Hide
          eepayne Eric Payne added a comment -

          Thanks, Gera Shegalov. Would it be okay if I upmerged MAPREDUCE-5044.v06.patch and integrated it with the SignalContainerRequest that was added as part of YARN-445 and its children?

          Show
          eepayne Eric Payne added a comment - Thanks, Gera Shegalov . Would it be okay if I upmerged MAPREDUCE-5044 .v06.patch and integrated it with the SignalContainerRequest that was added as part of YARN-445 and its children?
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Eric Payne yes, thank you for doing this.

          Show
          jira.shegalov Gera Shegalov added a comment - Eric Payne yes, thank you for doing this.
          Hide
          eepayne Eric Payne added a comment -

          Thanks, Gera Shegalov for all of the work already done on this JIRA.

          I have upmerged the latest patch and integrated it with the SignalContainerRequest that was added as part of YARN-445 and its children.

          Ming Ma, Xuan Gong, Jason Lowe, Gera Shegalov, would you please take a look?

          I would like to see functionality in this JIRA implemented. We occasionally see containers time out, and it would be good if users could have direct feedback in the form of a jstack to help them debug their applications.

          IIUC, YARN-445 and its children put in place the infrastructure for a Client -> RM -> NM -> Container signal path. However, in order to automatically dump the jstack when a container times out, we still need an AM -> NM -> Container signal path. This JIRA (MAPREDUCE-5044 along with YARN-1515) adds this signal path along with the ability to send multiple signals per call.

          I think sending multiple signals per call could be split into a separate JIRA.

          Show
          eepayne Eric Payne added a comment - Thanks, Gera Shegalov for all of the work already done on this JIRA. I have upmerged the latest patch and integrated it with the SignalContainerRequest that was added as part of YARN-445 and its children. Ming Ma , Xuan Gong , Jason Lowe , Gera Shegalov , would you please take a look? I would like to see functionality in this JIRA implemented. We occasionally see containers time out, and it would be good if users could have direct feedback in the form of a jstack to help them debug their applications. IIUC, YARN-445 and its children put in place the infrastructure for a Client -> RM -> NM -> Container signal path. However, in order to automatically dump the jstack when a container times out, we still need an AM -> NM -> Container signal path. This JIRA ( MAPREDUCE-5044 along with YARN-1515 ) adds this signal path along with the ability to send multiple signals per call. I think sending multiple signals per call could be split into a separate JIRA.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 1m 35s Maven dependency ordering for branch
          +1 mvninstall 7m 7s trunk passed
          +1 compile 6m 46s trunk passed with JDK v1.8.0_72
          +1 compile 6m 54s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 7s trunk passed
          +1 mvnsite 2m 35s trunk passed
          +1 mvneclipse 1m 8s trunk passed
          +1 findbugs 5m 8s trunk passed
          +1 javadoc 2m 5s trunk passed with JDK v1.8.0_72
          +1 javadoc 4m 31s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 2m 9s the patch passed
          +1 compile 6m 5s the patch passed with JDK v1.8.0_72
          +1 cc 6m 5s the patch passed
          +1 javac 6m 5s the patch passed
          +1 compile 6m 44s the patch passed with JDK v1.7.0_95
          +1 cc 6m 44s the patch passed
          +1 javac 6m 44s the patch passed
          -1 checkstyle 1m 9s root: patch generated 1 new + 330 unchanged - 0 fixed = 331 total (was 330)
          +1 mvnsite 2m 29s the patch passed
          +1 mvneclipse 1m 7s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 6m 1s the patch passed
          +1 javadoc 2m 0s the patch passed with JDK v1.8.0_72
          +1 javadoc 4m 33s the patch passed with JDK v1.7.0_95
          +1 unit 0m 22s hadoop-yarn-api in the patch passed with JDK v1.8.0_72.
          +1 unit 1m 55s hadoop-yarn-common in the patch passed with JDK v1.8.0_72.
          +1 unit 8m 48s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_72.
          -1 unit 70m 52s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72.
          +1 unit 9m 1s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_72.
          +1 unit 0m 24s hadoop-yarn-api in the patch passed with JDK v1.7.0_95.
          +1 unit 2m 10s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
          +1 unit 9m 16s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95.
          -1 unit 72m 14s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 unit 9m 31s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          258m 30s



          Reason Tests
          JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788869/MAPREDUCE-5044.v07.local.patch
          JIRA Issue MAPREDUCE-5044
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 94f89c6cb2de 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d5abd29
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 1m 35s Maven dependency ordering for branch +1 mvninstall 7m 7s trunk passed +1 compile 6m 46s trunk passed with JDK v1.8.0_72 +1 compile 6m 54s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 7s trunk passed +1 mvnsite 2m 35s trunk passed +1 mvneclipse 1m 8s trunk passed +1 findbugs 5m 8s trunk passed +1 javadoc 2m 5s trunk passed with JDK v1.8.0_72 +1 javadoc 4m 31s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 9s the patch passed +1 compile 6m 5s the patch passed with JDK v1.8.0_72 +1 cc 6m 5s the patch passed +1 javac 6m 5s the patch passed +1 compile 6m 44s the patch passed with JDK v1.7.0_95 +1 cc 6m 44s the patch passed +1 javac 6m 44s the patch passed -1 checkstyle 1m 9s root: patch generated 1 new + 330 unchanged - 0 fixed = 331 total (was 330) +1 mvnsite 2m 29s the patch passed +1 mvneclipse 1m 7s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 6m 1s the patch passed +1 javadoc 2m 0s the patch passed with JDK v1.8.0_72 +1 javadoc 4m 33s the patch passed with JDK v1.7.0_95 +1 unit 0m 22s hadoop-yarn-api in the patch passed with JDK v1.8.0_72. +1 unit 1m 55s hadoop-yarn-common in the patch passed with JDK v1.8.0_72. +1 unit 8m 48s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_72. -1 unit 70m 52s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. +1 unit 9m 1s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_72. +1 unit 0m 24s hadoop-yarn-api in the patch passed with JDK v1.7.0_95. +1 unit 2m 10s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. +1 unit 9m 16s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95. -1 unit 72m 14s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 unit 9m 31s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 258m 30s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788869/MAPREDUCE-5044.v07.local.patch JIRA Issue MAPREDUCE-5044 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 94f89c6cb2de 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d5abd29 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6334/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          eepayne Eric Payne added a comment -

          The following tests are not failing for me in my environment. I ran several times with both JDK 1.7 and 1.8.

          Test Related JIRA?
          hadoop.yarn.server.resourcemanager.TestClientRMTokens YARN-4306/HADOOP-12687
          hadoop.yarn.server.resourcemanager.TestAMAuthorization YARN-4318/HADOOP-12687
          hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler YARN-4556
          Show
          eepayne Eric Payne added a comment - The following tests are not failing for me in my environment. I ran several times with both JDK 1.7 and 1.8. Test Related JIRA? hadoop.yarn.server.resourcemanager.TestClientRMTokens YARN-4306 / HADOOP-12687 hadoop.yarn.server.resourcemanager.TestAMAuthorization YARN-4318 / HADOOP-12687 hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler YARN-4556
          Hide
          eepayne Eric Payne added a comment -

          Ming Ma, Xuan Gong, Jason Lowe, Gera Shegalov, did you have a chance to look at this patch? I would really appreciate some feedback.

          Show
          eepayne Eric Payne added a comment - Ming Ma , Xuan Gong , Jason Lowe , Gera Shegalov , did you have a chance to look at this patch? I would really appreciate some feedback.
          Hide
          eepayne Eric Payne added a comment -

          Gera Shegalov, Ming Ma, Xuan Gong, Jason Lowe,
          Upmerged patch and attaching MAPREDUCE-5044.008.patch.

          Show
          eepayne Eric Payne added a comment - Gera Shegalov , Ming Ma , Xuan Gong , Jason Lowe , Upmerged patch and attaching MAPREDUCE-5044 .008.patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 13m 50s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 0m 49s Maven dependency ordering for branch
          +1 mvninstall 8m 36s trunk passed
          +1 compile 9m 13s trunk passed with JDK v1.8.0_77
          +1 compile 8m 49s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 23s trunk passed
          +1 mvnsite 3m 1s trunk passed
          +1 mvneclipse 1m 21s trunk passed
          +1 findbugs 6m 10s trunk passed
          +1 javadoc 2m 46s trunk passed with JDK v1.8.0_77
          +1 javadoc 5m 26s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 2m 41s the patch passed
          +1 compile 9m 20s the patch passed with JDK v1.8.0_77
          +1 cc 9m 20s the patch passed
          +1 javac 9m 20s the patch passed
          +1 compile 8m 52s the patch passed with JDK v1.7.0_95
          +1 cc 8m 52s the patch passed
          +1 javac 8m 52s the patch passed
          -1 checkstyle 1m 22s root: patch generated 1 new + 324 unchanged - 0 fixed = 325 total (was 324)
          +1 mvnsite 3m 4s the patch passed
          +1 mvneclipse 1m 19s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 7m 25s the patch passed
          +1 javadoc 2m 47s the patch passed with JDK v1.8.0_77
          +1 javadoc 5m 27s the patch passed with JDK v1.7.0_95
          +1 unit 0m 31s hadoop-yarn-api in the patch passed with JDK v1.8.0_77.
          +1 unit 2m 33s hadoop-yarn-common in the patch passed with JDK v1.8.0_77.
          +1 unit 10m 23s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_77.
          -1 unit 81m 11s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          +1 unit 10m 58s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 31s hadoop-yarn-api in the patch passed with JDK v1.7.0_95.
          +1 unit 2m 41s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
          +1 unit 10m 38s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95.
          -1 unit 82m 51s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          -1 unit 12m 38s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 42s Patch does not generate ASF License warnings.
          321m 53s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.mapreduce.v2.app.job.impl.TestJobImpl
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799346/MAPREDUCE-5044.008.patch
          JIRA Issue MAPREDUCE-5044
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux eba77878ecbe 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 4770037
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 50s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 0m 49s Maven dependency ordering for branch +1 mvninstall 8m 36s trunk passed +1 compile 9m 13s trunk passed with JDK v1.8.0_77 +1 compile 8m 49s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 23s trunk passed +1 mvnsite 3m 1s trunk passed +1 mvneclipse 1m 21s trunk passed +1 findbugs 6m 10s trunk passed +1 javadoc 2m 46s trunk passed with JDK v1.8.0_77 +1 javadoc 5m 26s trunk passed with JDK v1.7.0_95 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 2m 41s the patch passed +1 compile 9m 20s the patch passed with JDK v1.8.0_77 +1 cc 9m 20s the patch passed +1 javac 9m 20s the patch passed +1 compile 8m 52s the patch passed with JDK v1.7.0_95 +1 cc 8m 52s the patch passed +1 javac 8m 52s the patch passed -1 checkstyle 1m 22s root: patch generated 1 new + 324 unchanged - 0 fixed = 325 total (was 324) +1 mvnsite 3m 4s the patch passed +1 mvneclipse 1m 19s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 7m 25s the patch passed +1 javadoc 2m 47s the patch passed with JDK v1.8.0_77 +1 javadoc 5m 27s the patch passed with JDK v1.7.0_95 +1 unit 0m 31s hadoop-yarn-api in the patch passed with JDK v1.8.0_77. +1 unit 2m 33s hadoop-yarn-common in the patch passed with JDK v1.8.0_77. +1 unit 10m 23s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_77. -1 unit 81m 11s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. +1 unit 10m 58s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_77. +1 unit 0m 31s hadoop-yarn-api in the patch passed with JDK v1.7.0_95. +1 unit 2m 41s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. +1 unit 10m 38s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95. -1 unit 82m 51s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. -1 unit 12m 38s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 42s Patch does not generate ASF License warnings. 321m 53s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.mapreduce.v2.app.job.impl.TestJobImpl JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799346/MAPREDUCE-5044.008.patch JIRA Issue MAPREDUCE-5044 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux eba77878ecbe 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4770037 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6446/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          eepayne Eric Payne added a comment -

          Jason Lowe, Gera Shegalov, Ming Ma, Xuan Gong,
          Patch 008 was no longer applying to trunk. I upmerged the patch and attaching MAPREDUCE-5044.009.patch. Can I please ask one of you to look at it?

          Show
          eepayne Eric Payne added a comment - Jason Lowe , Gera Shegalov , Ming Ma , Xuan Gong , Patch 008 was no longer applying to trunk. I upmerged the patch and attaching MAPREDUCE-5044 .009.patch. Can I please ask one of you to look at it?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 0m 43s Maven dependency ordering for branch
          +1 mvninstall 7m 40s trunk passed
          +1 compile 7m 10s trunk passed with JDK v1.8.0_91
          +1 compile 7m 20s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 38s trunk passed
          +1 mvnsite 2m 44s trunk passed
          +1 mvneclipse 1m 12s trunk passed
          -1 findbugs 1m 14s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in trunk has 1 extant Findbugs warnings.
          +1 javadoc 2m 16s trunk passed with JDK v1.8.0_91
          +1 javadoc 4m 48s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 2m 20s the patch passed
          +1 compile 7m 9s the patch passed with JDK v1.8.0_91
          -1 cc 9m 46s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 1 new + 10 unchanged - 1 fixed = 11 total (was 11)
          +1 cc 7m 9s the patch passed
          +1 javac 7m 9s the patch passed
          +1 compile 7m 45s the patch passed with JDK v1.7.0_95
          +1 cc 7m 45s the patch passed
          +1 javac 7m 45s the patch passed
          -1 checkstyle 1m 35s root: patch generated 2 new + 496 unchanged - 0 fixed = 498 total (was 496)
          +1 mvnsite 2m 43s the patch passed
          +1 mvneclipse 1m 11s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 6m 2s the patch passed
          -1 javadoc 3m 42s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_91 with JDK v1.8.0_91 generated 6 new + 5406 unchanged - 0 fixed = 5412 total (was 5406)
          +1 javadoc 2m 31s the patch passed with JDK v1.8.0_91
          +1 javadoc 4m 29s the patch passed with JDK v1.7.0_95
          +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.8.0_91.
          +1 unit 2m 17s hadoop-yarn-common in the patch passed with JDK v1.8.0_91.
          +1 unit 11m 37s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91.
          -1 unit 29m 44s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
          +1 unit 8m 17s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 26s hadoop-yarn-api in the patch passed with JDK v1.7.0_95.
          +1 unit 2m 24s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
          +1 unit 11m 49s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95.
          -1 unit 30m 53s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 unit 8m 50s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 24s Patch does not generate ASF License warnings.
          186m 27s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804245/MAPREDUCE-5044.009.patch
          JIRA Issue MAPREDUCE-5044
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 9a6d9ca1d576 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 576e2d1
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-warnings.html
          cc root-jdk1.8.0_91: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/diff-compile-cc-root-jdk1.8.0_91.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/diff-checkstyle-root.txt
          javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_91: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 0m 43s Maven dependency ordering for branch +1 mvninstall 7m 40s trunk passed +1 compile 7m 10s trunk passed with JDK v1.8.0_91 +1 compile 7m 20s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 38s trunk passed +1 mvnsite 2m 44s trunk passed +1 mvneclipse 1m 12s trunk passed -1 findbugs 1m 14s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in trunk has 1 extant Findbugs warnings. +1 javadoc 2m 16s trunk passed with JDK v1.8.0_91 +1 javadoc 4m 48s trunk passed with JDK v1.7.0_95 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 2m 20s the patch passed +1 compile 7m 9s the patch passed with JDK v1.8.0_91 -1 cc 9m 46s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 1 new + 10 unchanged - 1 fixed = 11 total (was 11) +1 cc 7m 9s the patch passed +1 javac 7m 9s the patch passed +1 compile 7m 45s the patch passed with JDK v1.7.0_95 +1 cc 7m 45s the patch passed +1 javac 7m 45s the patch passed -1 checkstyle 1m 35s root: patch generated 2 new + 496 unchanged - 0 fixed = 498 total (was 496) +1 mvnsite 2m 43s the patch passed +1 mvneclipse 1m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 6m 2s the patch passed -1 javadoc 3m 42s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_91 with JDK v1.8.0_91 generated 6 new + 5406 unchanged - 0 fixed = 5412 total (was 5406) +1 javadoc 2m 31s the patch passed with JDK v1.8.0_91 +1 javadoc 4m 29s the patch passed with JDK v1.7.0_95 +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.8.0_91. +1 unit 2m 17s hadoop-yarn-common in the patch passed with JDK v1.8.0_91. +1 unit 11m 37s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91. -1 unit 29m 44s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. +1 unit 8m 17s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_91. +1 unit 0m 26s hadoop-yarn-api in the patch passed with JDK v1.7.0_95. +1 unit 2m 24s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. +1 unit 11m 49s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95. -1 unit 30m 53s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 unit 8m 50s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 186m 27s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804245/MAPREDUCE-5044.009.patch JIRA Issue MAPREDUCE-5044 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 9a6d9ca1d576 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 576e2d1 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-warnings.html cc root-jdk1.8.0_91: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/diff-compile-cc-root-jdk1.8.0_91.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/diff-checkstyle-root.txt javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_91: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6501/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          mingma Ming Ma added a comment -

          Eric Payne, my apologies for the delay.

          • There was some discussion about combining signalContainer and stopContainers so that stopContainer is just a special case for signalContainer. And to support the "SIGTERM + delay + SIGKILL" used in stopContainers, we then need an ordered list of commands, thus the need for signalContainers. We don't need to deal with that at this point. But it might be useful to rename signalContainer to signalContainers so that we don't need to modify the API later, which means some new structure like SignalContainersRequest. What is your take?
          • ContainerManagerImpl. It might be cleaner to abstract the common signal container code to a function used for both AM -> NM and RM -> NM cases.
          • TaskAttemptImpl#PreemptedTransition. Given it is called only when the attempt is preempted, event.getType() == TaskAttemptEventType.TA_TIMED_OUT can be replaced by false.
          • It will be useful to add an end-to-end new unit test, which can be found in Gera's original patch.
          • Nit: ContainerLauncherImpl. Return value of getContainerManagementProtocol().signalContainer isn't used and can be removed.
          • Nit: ContainerLauncherEvent has indent format issue.
          Show
          mingma Ming Ma added a comment - Eric Payne , my apologies for the delay. There was some discussion about combining signalContainer and stopContainers so that stopContainer is just a special case for signalContainer. And to support the "SIGTERM + delay + SIGKILL" used in stopContainers, we then need an ordered list of commands, thus the need for signalContainers. We don't need to deal with that at this point. But it might be useful to rename signalContainer to signalContainers so that we don't need to modify the API later, which means some new structure like SignalContainersRequest . What is your take? ContainerManagerImpl. It might be cleaner to abstract the common signal container code to a function used for both AM -> NM and RM -> NM cases. TaskAttemptImpl#PreemptedTransition. Given it is called only when the attempt is preempted, event.getType() == TaskAttemptEventType.TA_TIMED_OUT can be replaced by false . It will be useful to add an end-to-end new unit test, which can be found in Gera's original patch. Nit: ContainerLauncherImpl. Return value of getContainerManagementProtocol().signalContainer isn't used and can be removed. Nit: ContainerLauncherEvent has indent format issue.
          Hide
          eepayne Eric Payne added a comment -

          Ming Ma, thank you very much for the comments. I have one question:

          • ... it might be useful to rename signalContainer to signalContainers so that we don't need to modify the API later, which means some new structure like SignalContainersRequest. What is your take?

          I would rather not rename signalContainer to signalContainers because signalContainers sounds to me like the purpose is to send one signal to multiple containers rather than to send multiple signals to one container. Calling it signalsContainer (plural signals) also sounds awkward. So, I think signalContainer is the best option.

          Regarding SignalContainerRequest, if we want the signalContainer API to be fully compatible with sending multiple signals, I think SignalContainerRequest would need to add an interface for SignalContainerRequest#newInstance that included both pause and a list of signals. Maybe something like this:

          public static SignalContainerRequest newInstance(ContainerId containerId,
              int pause, Iterable<SignalContainerCommand> signals) {
          ...
          }
          

          I think it would be best to add that interface to SignalContainerRequest in the future when we are ready to implement the rest of the "sending multiple signals" feature. Thoughts?

          Show
          eepayne Eric Payne added a comment - Ming Ma , thank you very much for the comments. I have one question: ... it might be useful to rename signalContainer to signalContainers so that we don't need to modify the API later, which means some new structure like SignalContainersRequest. What is your take? I would rather not rename signalContainer to signalContainers because signalContainers sounds to me like the purpose is to send one signal to multiple containers rather than to send multiple signals to one container. Calling it signalsContainer (plural signals ) also sounds awkward. So, I think signalContainer is the best option. Regarding SignalContainerRequest , if we want the signalContainer API to be fully compatible with sending multiple signals, I think SignalContainerRequest would need to add an interface for SignalContainerRequest#newInstance that included both pause and a list of signals. Maybe something like this: public static SignalContainerRequest newInstance(ContainerId containerId, int pause, Iterable<SignalContainerCommand> signals) { ... } I think it would be best to add that interface to SignalContainerRequest in the future when we are ready to implement the rest of the "sending multiple signals" feature. Thoughts?
          Hide
          mingma Ming Ma added a comment -

          Eric Payne I agree with your suggestion. Let us postpone it to a later time.

          • signalContainers was initially suggested as an ordered list of signalContainer. So it could include requests from the same container or requests from different containers. It is true that the only use case we know of so far is to include requests from the same container.
          • We also discussed introducing other commands besides linux signal, for example sleep command used to pause between signals, in that way, the new API could be just like
            public static SignalContainerRequest newInstance(ContainerId containerId,
                Iterable<SignalContainerCommand> signals) {
            ...
            }
            
          • Will the required in the protocol buffer definition create any issue if we do rolling upgrade from 2.8 to 2.9 and the 2.9 MR AM might send a list of SignalContainerCommandProto to 2.8 NM? Maybe 2.8 NM just discards the message, not a big deal. Regardless, that is a separate issue that we don't need to address it here.
          message SignalContainerRequestProto {
          ....
              required SignalContainerCommandProto command = 2;
          }
          
          Show
          mingma Ming Ma added a comment - Eric Payne I agree with your suggestion. Let us postpone it to a later time. signalContainers was initially suggested as an ordered list of signalContainer . So it could include requests from the same container or requests from different containers. It is true that the only use case we know of so far is to include requests from the same container. We also discussed introducing other commands besides linux signal, for example sleep command used to pause between signals, in that way, the new API could be just like public static SignalContainerRequest newInstance(ContainerId containerId, Iterable<SignalContainerCommand> signals) { ... } Will the required in the protocol buffer definition create any issue if we do rolling upgrade from 2.8 to 2.9 and the 2.9 MR AM might send a list of SignalContainerCommandProto to 2.8 NM? Maybe 2.8 NM just discards the message, not a big deal. Regardless, that is a separate issue that we don't need to address it here. message SignalContainerRequestProto { .... required SignalContainerCommandProto command = 2; }
          Hide
          eepayne Eric Payne added a comment -

          Ming Ma, thank you for your reply and explanation.

          • signalContainers was initially suggested as an ordered list of signalContainer. So it could include requests from the same container or requests from different containers. It is true that the only use case we know of so far is to include requests from the same container.

          In that case, do we want to call it something like signalsToContainers? I'm open for ideas.

          • Will the required in the protocol buffer definition create any issue if we do rolling upgrade from 2.8 to 2.9 and the 2.9 MR AM might send a list of SignalContainerCommandProto to 2.8 NM? Maybe 2.8 NM just discards the message, not a big deal. Regardless, that is a separate issue that we don't need to address it here.

          Yes, this is a concern and something we need to look into more deeply and keep in mind.

          Show
          eepayne Eric Payne added a comment - Ming Ma , thank you for your reply and explanation. signalContainers was initially suggested as an ordered list of signalContainer. So it could include requests from the same container or requests from different containers. It is true that the only use case we know of so far is to include requests from the same container. In that case, do we want to call it something like signalsToContainers ? I'm open for ideas. Will the required in the protocol buffer definition create any issue if we do rolling upgrade from 2.8 to 2.9 and the 2.9 MR AM might send a list of SignalContainerCommandProto to 2.8 NM? Maybe 2.8 NM just discards the message, not a big deal. Regardless, that is a separate issue that we don't need to address it here. Yes, this is a concern and something we need to look into more deeply and keep in mind.
          Hide
          mingma Ming Ma added a comment -

          In that case, do we want to call it something like signalsToContainers?

          Sounds good. signalsToContainers can take an array of SignalContainerRequest, each of which has a list of commands belonging to the same container. When we decide to add signalsToContainers later, deprecate signalToContainer and NM will still support signalToContainer until major upgrade. In that way, we don't need to fix required issue given only new signalsToContainers method will use list-based SignalContainerRequest.

          Show
          mingma Ming Ma added a comment - In that case, do we want to call it something like signalsToContainers? Sounds good. signalsToContainers can take an array of SignalContainerRequest , each of which has a list of commands belonging to the same container. When we decide to add signalsToContainers later, deprecate signalToContainer and NM will still support signalToContainer until major upgrade. In that way, we don't need to fix required issue given only new signalsToContainers method will use list-based SignalContainerRequest .
          Hide
          eepayne Eric Payne added a comment -

          Thank you very much, Ming Ma, for your review and suggestions:

          • rename signalContainer to signalToContainer

          Done.

          • ContainerManagerImpl. It might be cleaner to abstract the common signal container code to a function used for both AM -> NM and RM -> NM cases.

          Done.

          • TaskAttemptImpl#PreemptedTransition. Given it is called only when the attempt is preempted, event.getType() == TaskAttemptEventType.TA_TIMED_OUT can be replaced by false.

          Very true. Good catch.

          • It will be useful to add an end-to-end new unit test, which can be found in Gera's original patch.

          Done.

          • Nit: ContainerLauncherImpl. Return value of getContainerManagementProtocol().signalContainer isn't used and can be removed.

          Done.

          • Nit: ContainerLauncherEvent has indent format issue.

          Done.

          Show
          eepayne Eric Payne added a comment - Thank you very much, Ming Ma , for your review and suggestions: rename signalContainer to signalToContainer Done. ContainerManagerImpl . It might be cleaner to abstract the common signal container code to a function used for both AM -> NM and RM -> NM cases. Done. TaskAttemptImpl#PreemptedTransition . Given it is called only when the attempt is preempted, event.getType() == TaskAttemptEventType.TA_TIMED_OUT can be replaced by false. Very true. Good catch. It will be useful to add an end-to-end new unit test, which can be found in Gera's original patch. Done. Nit: ContainerLauncherImpl . Return value of getContainerManagementProtocol().signalContainer isn't used and can be removed. Done. Nit: ContainerLauncherEvent has indent format issue. Done.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 3m 16s Docker failed to build yetus/hadoop:2c91fd8.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805487/MAPREDUCE-5044.010.patch
          JIRA Issue MAPREDUCE-5044
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6523/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 3m 16s Docker failed to build yetus/hadoop:2c91fd8. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805487/MAPREDUCE-5044.010.patch JIRA Issue MAPREDUCE-5044 Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6523/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 9 new or modified test files.
          0 mvndep 0m 31s Maven dependency ordering for branch
          +1 mvninstall 6m 31s trunk passed
          +1 compile 6m 36s trunk passed
          +1 checkstyle 1m 26s trunk passed
          +1 mvnsite 2m 45s trunk passed
          +1 mvneclipse 1m 8s trunk passed
          +1 findbugs 4m 26s trunk passed
          +1 javadoc 2m 16s trunk passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 2m 18s the patch passed
          +1 compile 6m 32s the patch passed
          +1 cc 6m 32s the patch passed
          +1 javac 6m 32s the patch passed
          -1 checkstyle 1m 26s root: patch generated 21 new + 537 unchanged - 0 fixed = 558 total (was 537)
          +1 mvnsite 2m 42s the patch passed
          +1 mvneclipse 1m 8s the patch passed
          -1 whitespace 0m 0s The patch has 3 line(s) with tabs.
          +1 findbugs 5m 13s the patch passed
          -1 javadoc 3m 0s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 6 new + 5406 unchanged - 0 fixed = 5412 total (was 5406)
          +1 javadoc 2m 15s the patch passed
          +1 unit 0m 23s hadoop-yarn-api in the patch passed.
          +1 unit 2m 8s hadoop-yarn-common in the patch passed.
          +1 unit 11m 13s hadoop-yarn-server-nodemanager in the patch passed.
          -1 unit 29m 29s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 unit 8m 16s hadoop-mapreduce-client-app in the patch passed.
          -1 unit 100m 11s hadoop-mapreduce-client-jobclient in the patch failed.
          +1 asflicense 0m 34s Patch does not generate ASF License warnings.
          201m 15s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.mapreduce.v2.TestUberAM
            hadoop.mapred.TestMiniMRChildTask



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805487/MAPREDUCE-5044.010.patch
          JIRA Issue MAPREDUCE-5044
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 37a9d60b5ea1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 12eb5a7
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/whitespace-tabs.txt
          javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 9 new or modified test files. 0 mvndep 0m 31s Maven dependency ordering for branch +1 mvninstall 6m 31s trunk passed +1 compile 6m 36s trunk passed +1 checkstyle 1m 26s trunk passed +1 mvnsite 2m 45s trunk passed +1 mvneclipse 1m 8s trunk passed +1 findbugs 4m 26s trunk passed +1 javadoc 2m 16s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 2m 18s the patch passed +1 compile 6m 32s the patch passed +1 cc 6m 32s the patch passed +1 javac 6m 32s the patch passed -1 checkstyle 1m 26s root: patch generated 21 new + 537 unchanged - 0 fixed = 558 total (was 537) +1 mvnsite 2m 42s the patch passed +1 mvneclipse 1m 8s the patch passed -1 whitespace 0m 0s The patch has 3 line(s) with tabs. +1 findbugs 5m 13s the patch passed -1 javadoc 3m 0s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 6 new + 5406 unchanged - 0 fixed = 5412 total (was 5406) +1 javadoc 2m 15s the patch passed +1 unit 0m 23s hadoop-yarn-api in the patch passed. +1 unit 2m 8s hadoop-yarn-common in the patch passed. +1 unit 11m 13s hadoop-yarn-server-nodemanager in the patch passed. -1 unit 29m 29s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 8m 16s hadoop-mapreduce-client-app in the patch passed. -1 unit 100m 11s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 0m 34s Patch does not generate ASF License warnings. 201m 15s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.mapreduce.v2.TestUberAM   hadoop.mapred.TestMiniMRChildTask Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805487/MAPREDUCE-5044.010.patch JIRA Issue MAPREDUCE-5044 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 37a9d60b5ea1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 12eb5a7 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/whitespace-tabs.txt javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6524/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          mingma Ming Ma added a comment -

          Thanks Eric Payne. Besides the checkstyle, whitespace and javadoc issues,

          • There is some commented-out code left after the function is moved to internalSignalToContainer.
          • Given signalContainer is renamed to signalToContainer for ContainerManagementProtocol, maybe better to fix that for ApplicationClientProtocol as well, as long as we agree to include this patch in 2.8.

          Otherwise, it looks good overall.

          Show
          mingma Ming Ma added a comment - Thanks Eric Payne . Besides the checkstyle, whitespace and javadoc issues, There is some commented-out code left after the function is moved to internalSignalToContainer . Given signalContainer is renamed to signalToContainer for ContainerManagementProtocol, maybe better to fix that for ApplicationClientProtocol as well, as long as we agree to include this patch in 2.8. Otherwise, it looks good overall.
          Hide
          eepayne Eric Payne added a comment -

          Thanks for the continuing review, Ming Ma. I made the suggested changes. Please find them in attachment MAPREDUCE-5044.011.patch.

          Show
          eepayne Eric Payne added a comment - Thanks for the continuing review, Ming Ma . I made the suggested changes. Please find them in attachment MAPREDUCE-5044 .011.patch .
          Hide
          eepayne Eric Payne added a comment -

          It looks like the pre-commit build faild:
          https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/console

          Slave went offline during the build
          ERROR: Connection was broken: java.io.IOException: Sorry, this connection is closed.

          Cancelling patch and re-Submitting patch

          Show
          eepayne Eric Payne added a comment - It looks like the pre-commit build faild: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/console Slave went offline during the build ERROR: Connection was broken: java.io.IOException: Sorry, this connection is closed. Cancelling patch and re-Submitting patch
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 14 new or modified test files.
          0 mvndep 0m 12s Maven dependency ordering for branch
          +1 mvninstall 6m 25s trunk passed
          +1 compile 6m 42s trunk passed
          +1 checkstyle 1m 38s trunk passed
          +1 mvnsite 3m 20s trunk passed
          +1 mvneclipse 1m 24s trunk passed
          +1 findbugs 5m 17s trunk passed
          +1 javadoc 2m 41s trunk passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 2m 47s the patch passed
          +1 compile 7m 12s the patch passed
          +1 cc 7m 12s the patch passed
          +1 javac 7m 12s the patch passed
          -1 checkstyle 1m 34s root: patch generated 22 new + 873 unchanged - 3 fixed = 895 total (was 876)
          +1 mvnsite 3m 20s the patch passed
          +1 mvneclipse 1m 24s the patch passed
          -1 whitespace 0m 0s The patch has 2 line(s) with tabs.
          +1 findbugs 6m 8s the patch passed
          -1 javadoc 3m 25s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 6 new + 5406 unchanged - 0 fixed = 5412 total (was 5406)
          +1 javadoc 2m 37s the patch passed
          +1 unit 0m 27s hadoop-yarn-api in the patch passed.
          +1 unit 2m 13s hadoop-yarn-common in the patch passed.
          +1 unit 11m 14s hadoop-yarn-server-nodemanager in the patch passed.
          -1 unit 30m 9s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 unit 68m 7s hadoop-yarn-client in the patch failed.
          +1 unit 8m 33s hadoop-mapreduce-client-app in the patch passed.
          -1 unit 127m 42s hadoop-mapreduce-client-jobclient in the patch failed.
          +1 asflicense 0m 29s Patch does not generate ASF License warnings.
          303m 27s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestRMAdminService
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.client.TestGetGroups
            hadoop.yarn.client.api.impl.TestDistributedScheduling
            hadoop.yarn.client.api.impl.TestAMRMProxy
            hadoop.mapred.TestMiniMRChildTask
            hadoop.mapred.TestMRIntermediateDataEncryption
            hadoop.mapreduce.v2.TestUberAM
          Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient
            org.apache.hadoop.mapred.TestMROpportunisticMaps



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806156/MAPREDUCE-5044.011.patch
          JIRA Issue MAPREDUCE-5044
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 4791a35479fd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9a31e5d
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/whitespace-tabs.txt
          javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 14 new or modified test files. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 6m 25s trunk passed +1 compile 6m 42s trunk passed +1 checkstyle 1m 38s trunk passed +1 mvnsite 3m 20s trunk passed +1 mvneclipse 1m 24s trunk passed +1 findbugs 5m 17s trunk passed +1 javadoc 2m 41s trunk passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 2m 47s the patch passed +1 compile 7m 12s the patch passed +1 cc 7m 12s the patch passed +1 javac 7m 12s the patch passed -1 checkstyle 1m 34s root: patch generated 22 new + 873 unchanged - 3 fixed = 895 total (was 876) +1 mvnsite 3m 20s the patch passed +1 mvneclipse 1m 24s the patch passed -1 whitespace 0m 0s The patch has 2 line(s) with tabs. +1 findbugs 6m 8s the patch passed -1 javadoc 3m 25s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 6 new + 5406 unchanged - 0 fixed = 5412 total (was 5406) +1 javadoc 2m 37s the patch passed +1 unit 0m 27s hadoop-yarn-api in the patch passed. +1 unit 2m 13s hadoop-yarn-common in the patch passed. +1 unit 11m 14s hadoop-yarn-server-nodemanager in the patch passed. -1 unit 30m 9s hadoop-yarn-server-resourcemanager in the patch failed. -1 unit 68m 7s hadoop-yarn-client in the patch failed. +1 unit 8m 33s hadoop-mapreduce-client-app in the patch passed. -1 unit 127m 42s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 0m 29s Patch does not generate ASF License warnings. 303m 27s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMAdminService   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.api.impl.TestDistributedScheduling   hadoop.yarn.client.api.impl.TestAMRMProxy   hadoop.mapred.TestMiniMRChildTask   hadoop.mapred.TestMRIntermediateDataEncryption   hadoop.mapreduce.v2.TestUberAM Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient   org.apache.hadoop.mapred.TestMROpportunisticMaps Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806156/MAPREDUCE-5044.011.patch JIRA Issue MAPREDUCE-5044 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 4791a35479fd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9a31e5d Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/whitespace-tabs.txt javadoc hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6528/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Thanks for picking up this JIRA Eric Payne, assigning it to you!

          Show
          jira.shegalov Gera Shegalov added a comment - Thanks for picking up this JIRA Eric Payne , assigning it to you!
          Hide
          eepayne Eric Payne added a comment -

          Of the unit tests that failed in the precommit build, all pass for me in my local build environment except for 2:

          • TestMiniMRChildTask fails in trunk with or without MAPREDUCE-5044.011.patch
          • TestUberAM succeeds in trunk and fails with MAPREDUCE-5044.011.patch.
            This is because TestUberAM extends TestMRJobs, to which I added the test testThreadDumpOnTaskTimeout. TestMRJobs#testThreadDumpOnTaskTimeout is having issues. I will fix and upload a new patch.
          Show
          eepayne Eric Payne added a comment - Of the unit tests that failed in the precommit build, all pass for me in my local build environment except for 2: TestMiniMRChildTask fails in trunk with or without MAPREDUCE-5044 .011.patch TestUberAM succeeds in trunk and fails with MAPREDUCE-5044 .011.patch . This is because TestUberAM extends TestMRJobs , to which I added the test testThreadDumpOnTaskTimeout . TestMRJobs#testThreadDumpOnTaskTimeout is having issues. I will fix and upload a new patch.
          Hide
          eepayne Eric Payne added a comment -

          The new test case (TestMRJobs#testThreadDumpOnTaskTimeout) when run with TestUberAM, detected that timeout did not cause a thread dump within an uber AM. So, I added code in LocalContainerLauncher in the latest patch (MAPREDUCE-5044.012.patch) to handle the timeout event.

          Instead of having the uber AM connect to the NM which would then send the QUIT signal back to the uber AM, I chose to dump the stack directly from the uber AM. I chose to use ThreadMXBean#dumpAllThreads even though there was already a Hadoop ReflwctionUtils#printThreadInfo method which would create a dump. The reason is because the output of ThreadMXBean#dumpAllThreads much more closely resembles the standard thread stack dump than does the output of ReflwctionUtils#printThreadInfo.

          Show
          eepayne Eric Payne added a comment - The new test case ( TestMRJobs#testThreadDumpOnTaskTimeout ) when run with TestUberAM , detected that timeout did not cause a thread dump within an uber AM. So, I added code in LocalContainerLauncher in the latest patch ( MAPREDUCE-5044 .012.patch ) to handle the timeout event. Instead of having the uber AM connect to the NM which would then send the QUIT signal back to the uber AM, I chose to dump the stack directly from the uber AM. I chose to use ThreadMXBean#dumpAllThreads even though there was already a Hadoop ReflwctionUtils#printThreadInfo method which would create a dump. The reason is because the output of ThreadMXBean#dumpAllThreads much more closely resembles the standard thread stack dump than does the output of ReflwctionUtils#printThreadInfo .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 25s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 14 new or modified test files.
          0 mvndep 0m 12s Maven dependency ordering for branch
          +1 mvninstall 7m 44s trunk passed
          +1 compile 8m 15s trunk passed
          +1 checkstyle 1m 45s trunk passed
          +1 mvnsite 3m 15s trunk passed
          +1 mvneclipse 1m 20s trunk passed
          +1 findbugs 5m 0s trunk passed
          +1 javadoc 2m 3s trunk passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 3m 14s the patch passed
          +1 compile 7m 39s the patch passed
          +1 cc 7m 39s the patch passed
          +1 javac 7m 39s the patch passed
          -1 checkstyle 1m 45s root: The patch generated 25 new + 879 unchanged - 3 fixed = 904 total (was 882)
          +1 mvnsite 3m 27s the patch passed
          +1 mvneclipse 1m 20s the patch passed
          -1 whitespace 0m 0s The patch 2 line(s) with tabs.
          +1 findbugs 6m 35s the patch passed
          +1 javadoc 2m 12s the patch passed
          +1 unit 0m 26s hadoop-yarn-api in the patch passed.
          +1 unit 2m 8s hadoop-yarn-common in the patch passed.
          +1 unit 10m 46s hadoop-yarn-server-nodemanager in the patch passed.
          -1 unit 32m 36s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 unit 68m 8s hadoop-yarn-client in the patch failed.
          -1 unit 9m 20s hadoop-mapreduce-client-app in the patch failed.
          -1 unit 115m 30s hadoop-mapreduce-client-jobclient in the patch failed.
          +1 asflicense 0m 30s The patch does not generate ASF License warnings.
          297m 21s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.client.cli.TestLogsCLI
            hadoop.yarn.client.TestGetGroups
            hadoop.yarn.client.api.impl.TestAMRMProxy
            hadoop.yarn.client.api.impl.TestDistributedScheduling
            hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
            hadoop.mapred.TestMiniMRChildTask
          Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807541/MAPREDUCE-5044.012.patch
          JIRA Issue MAPREDUCE-5044
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 769b35e62bd6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 16b1cc7
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 14 new or modified test files. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 7m 44s trunk passed +1 compile 8m 15s trunk passed +1 checkstyle 1m 45s trunk passed +1 mvnsite 3m 15s trunk passed +1 mvneclipse 1m 20s trunk passed +1 findbugs 5m 0s trunk passed +1 javadoc 2m 3s trunk passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 3m 14s the patch passed +1 compile 7m 39s the patch passed +1 cc 7m 39s the patch passed +1 javac 7m 39s the patch passed -1 checkstyle 1m 45s root: The patch generated 25 new + 879 unchanged - 3 fixed = 904 total (was 882) +1 mvnsite 3m 27s the patch passed +1 mvneclipse 1m 20s the patch passed -1 whitespace 0m 0s The patch 2 line(s) with tabs. +1 findbugs 6m 35s the patch passed +1 javadoc 2m 12s the patch passed +1 unit 0m 26s hadoop-yarn-api in the patch passed. +1 unit 2m 8s hadoop-yarn-common in the patch passed. +1 unit 10m 46s hadoop-yarn-server-nodemanager in the patch passed. -1 unit 32m 36s hadoop-yarn-server-resourcemanager in the patch failed. -1 unit 68m 8s hadoop-yarn-client in the patch failed. -1 unit 9m 20s hadoop-mapreduce-client-app in the patch failed. -1 unit 115m 30s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 297m 21s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.client.cli.TestLogsCLI   hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.api.impl.TestAMRMProxy   hadoop.yarn.client.api.impl.TestDistributedScheduling   hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt   hadoop.mapred.TestMiniMRChildTask Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807541/MAPREDUCE-5044.012.patch JIRA Issue MAPREDUCE-5044 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 769b35e62bd6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 16b1cc7 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6531/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          eepayne Eric Payne added a comment -

          I looked at the unit test failures from the pre-commit build. The all succeed in my local build environment except for TestYarnClient, which fails intermittently in trunk, both with and without this patch.

          Ming Ma, when you have some time, please have a look at the latest patch.

          Show
          eepayne Eric Payne added a comment - I looked at the unit test failures from the pre-commit build. The all succeed in my local build environment except for TestYarnClient, which fails intermittently in trunk, both with and without this patch. Ming Ma , when you have some time, please have a look at the latest patch.
          Hide
          aw Allen Wittenauer added a comment -

          Some (more) of those checkstyle errors should be fixed.

          Show
          aw Allen Wittenauer added a comment - Some (more) of those checkstyle errors should be fixed.
          Hide
          eepayne Eric Payne added a comment -

          Thanks Allen Wittenauer. I will look into those warnings.

          Show
          eepayne Eric Payne added a comment - Thanks Allen Wittenauer . I will look into those warnings.
          Hide
          eepayne Eric Payne added a comment -

          MAPREDUCE-5044.013.patch addresses most of the checkstyle warnings from the previous pre-commit build. The one exception is for those flagged in TestMRJobs.java. It is triggering the alert because of an inner assignments in the following for loop:

          for (String line; (line = syslogReader.readLine()) != null; ) {
          ...
          }
          

          The code could be changed to something like the following, but I think that would be more awkward:

          String line = syslogReader.readLine();
          for (String line; line != null; ) {
          ...
          line = syslogReader.readLine();
          }
          
          Show
          eepayne Eric Payne added a comment - MAPREDUCE-5044 .013.patch addresses most of the checkstyle warnings from the previous pre-commit build. The one exception is for those flagged in TestMRJobs.java . It is triggering the alert because of an inner assignments in the following for loop: for ( String line; (line = syslogReader.readLine()) != null ; ) { ... } The code could be changed to something like the following, but I think that would be more awkward: String line = syslogReader.readLine(); for ( String line; line != null ; ) { ... line = syslogReader.readLine(); }
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 25s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 14 new or modified test files.
          0 mvndep 0m 12s Maven dependency ordering for branch
          +1 mvninstall 6m 42s trunk passed
          +1 compile 6m 56s trunk passed
          +1 checkstyle 1m 32s trunk passed
          +1 mvnsite 3m 6s trunk passed
          +1 mvneclipse 1m 21s trunk passed
          -1 findbugs 0m 59s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in trunk has 1 extant Findbugs warnings.
          +1 javadoc 2m 0s trunk passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 2m 28s the patch passed
          +1 compile 6m 11s the patch passed
          +1 cc 6m 11s the patch passed
          +1 javac 6m 11s the patch passed
          -1 checkstyle 1m 31s root: The patch generated 4 new + 881 unchanged - 3 fixed = 885 total (was 884)
          +1 mvnsite 2m 59s the patch passed
          +1 mvneclipse 1m 21s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 5m 32s the patch passed
          +1 javadoc 1m 59s the patch passed
          +1 unit 0m 23s hadoop-yarn-api in the patch passed.
          +1 unit 2m 7s hadoop-yarn-common in the patch passed.
          +1 unit 10m 48s hadoop-yarn-server-nodemanager in the patch passed.
          -1 unit 35m 27s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 unit 67m 45s hadoop-yarn-client in the patch failed.
          -1 unit 0m 21s hadoop-mapreduce-client-app in the patch failed.
          -1 unit 0m 12s hadoop-mapreduce-client-jobclient in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          168m 29s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.client.api.impl.TestAMRMProxy
            hadoop.yarn.client.api.impl.TestDistributedScheduling
            hadoop.yarn.client.TestGetGroups
            hadoop.yarn.client.cli.TestLogsCLI
          Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808064/MAPREDUCE-5044.013.patch
          JIRA Issue MAPREDUCE-5044
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 53c52b9d38ec 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f10ebc6
          Default Java 1.8.0_91
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-warnings.html
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 14 new or modified test files. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 6m 42s trunk passed +1 compile 6m 56s trunk passed +1 checkstyle 1m 32s trunk passed +1 mvnsite 3m 6s trunk passed +1 mvneclipse 1m 21s trunk passed -1 findbugs 0m 59s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in trunk has 1 extant Findbugs warnings. +1 javadoc 2m 0s trunk passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 2m 28s the patch passed +1 compile 6m 11s the patch passed +1 cc 6m 11s the patch passed +1 javac 6m 11s the patch passed -1 checkstyle 1m 31s root: The patch generated 4 new + 881 unchanged - 3 fixed = 885 total (was 884) +1 mvnsite 2m 59s the patch passed +1 mvneclipse 1m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 32s the patch passed +1 javadoc 1m 59s the patch passed +1 unit 0m 23s hadoop-yarn-api in the patch passed. +1 unit 2m 7s hadoop-yarn-common in the patch passed. +1 unit 10m 48s hadoop-yarn-server-nodemanager in the patch passed. -1 unit 35m 27s hadoop-yarn-server-resourcemanager in the patch failed. -1 unit 67m 45s hadoop-yarn-client in the patch failed. -1 unit 0m 21s hadoop-mapreduce-client-app in the patch failed. -1 unit 0m 12s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 168m 29s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.client.api.impl.TestAMRMProxy   hadoop.yarn.client.api.impl.TestDistributedScheduling   hadoop.yarn.client.TestGetGroups   hadoop.yarn.client.cli.TestLogsCLI Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808064/MAPREDUCE-5044.013.patch JIRA Issue MAPREDUCE-5044 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 53c52b9d38ec 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f10ebc6 Default Java 1.8.0_91 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-warnings.html checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6534/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          eepayne Eric Payne added a comment -
          • FindBugs warning is not related. It pertains to org.apache.hadoop.yarn.api.records.ResourceRequest / ResourceRequest.java:[line 361], which was not changed by this patch.
          • Checkstyle warnings are as I expected (see my comment, above).
          • Unit test failures all pass in my local environment for TestLogsCLI, which intermittently fails both with and without this patch, and TestYarnClient, which fails consistently both with and without the patch.
          Show
          eepayne Eric Payne added a comment - FindBugs warning is not related. It pertains to org.apache.hadoop.yarn.api.records.ResourceRequest / ResourceRequest.java: [line 361] , which was not changed by this patch. Checkstyle warnings are as I expected (see my comment, above). Unit test failures all pass in my local environment for TestLogsCLI , which intermittently fails both with and without this patch, and TestYarnClient , which fails consistently both with and without the patch.
          Hide
          mingma Ming Ma added a comment -

          +1 on the latest patch. Thanks Eric Payne. The patch doesn't resolve automatically for branch-2 and 2.8. It is straightforward and I will resolve it for those two branches.

          Show
          mingma Ming Ma added a comment - +1 on the latest patch. Thanks Eric Payne . The patch doesn't resolve automatically for branch-2 and 2.8. It is straightforward and I will resolve it for those two branches.
          Hide
          eepayne Eric Payne added a comment -

          The patch doesn't resolve automatically for branch-2 and 2.8. It is straightforward and I will resolve it for those two branches.

          Ming Ma,
          I did see that, but I was hoping it was straightforward enough that it didn't need a separate patch. Thanks for doing the extra work for the cherry-pick.

          Show
          eepayne Eric Payne added a comment - The patch doesn't resolve automatically for branch-2 and 2.8. It is straightforward and I will resolve it for those two branches. Ming Ma , I did see that, but I was hoping it was straightforward enough that it didn't need a separate patch. Thanks for doing the extra work for the cherry-pick.
          Hide
          mingma Ming Ma added a comment -

          I have committed the patch to trunk, branch-2 and branch-2.8. Thank you Eric Payne and Gera Shegalov for the contribution and Vinod Kumar Vavilapalli Jason Lowe and Allen Wittenauer for the review.

          Show
          mingma Ming Ma added a comment - I have committed the patch to trunk, branch-2 and branch-2.8. Thank you Eric Payne and Gera Shegalov for the contribution and Vinod Kumar Vavilapalli Jason Lowe and Allen Wittenauer for the review.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9915 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9915/)
          MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout (mingma: rev 4a1cedc010d3fa1d8ef3f2773ca12acadfee5ba5)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestSignalContainer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeManager.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/SignalContainerResponse.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerResourceIncreaseRPC.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/MockResourceManagerFacade.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ContainerManagementProtocolPBServiceImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestYarnClient.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocolPB.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/containermanagement_protocol.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherEvent.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocol.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/LocalContainerLauncher.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagementProtocolPBClientImpl.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncherImpl.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9915 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9915/ ) MAPREDUCE-5044 . Have AM trigger jstack on task attempts that timeout (mingma: rev 4a1cedc010d3fa1d8ef3f2773ca12acadfee5ba5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestSignalContainer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeManager.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/SignalContainerResponse.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerResourceIncreaseRPC.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/MockResourceManagerFacade.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ContainerManagementProtocolPBServiceImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestYarnClient.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocolPB.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/containermanagement_protocol.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherEvent.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocol.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/LocalContainerLauncher.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagementProtocolPBClientImpl.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncherImpl.java

            People

            • Assignee:
              eepayne Eric Payne
              Reporter:
              jlowe Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              24 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development