Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.1, 0.24.0
    • Fix Version/s: 0.23.1
    • Component/s: applicationmaster, mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846.
      Show
      Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 .

      Description

      It looks like recovering from an RM AM dieing works very well on a single failure. But if it fails multiple times we appear to get into a live lock situation.

      yarn jar hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-*-SNAPSHOT.jar wordcount -Dyarn.app.mapreduce.am.log.level=DEBUG -Dmapreduce.job.reduces=30 input output
      12/02/03 21:06:57 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
      12/02/03 21:06:57 WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
      12/02/03 21:06:57 INFO input.FileInputFormat: Total input paths to process : 17
      12/02/03 21:06:57 INFO util.NativeCodeLoader: Loaded the native-hadoop library
      12/02/03 21:06:57 WARN snappy.LoadSnappy: Snappy native library not loaded
      12/02/03 21:06:57 INFO mapreduce.JobSubmitter: number of splits:17
      12/02/03 21:06:57 INFO mapred.ResourceMgrDelegate: Submitted application application_1328302034486_0003 to ResourceManager at HOST/IP:8040
      12/02/03 21:06:57 INFO mapreduce.Job: The url to track the job: http://HOST:8088/proxy/application_1328302034486_0003/
      12/02/03 21:06:57 INFO mapreduce.Job: Running job: job_1328302034486_0003
      12/02/03 21:07:03 INFO mapreduce.Job: Job job_1328302034486_0003 running in uber mode : false
      12/02/03 21:07:03 INFO mapreduce.Job:  map 0% reduce 0%
      12/02/03 21:07:09 INFO mapreduce.Job:  map 5% reduce 0%
      12/02/03 21:07:10 INFO mapreduce.Job:  map 17% reduce 0%
      #KILLED AM with kill -9 here
      12/02/03 21:07:16 INFO mapreduce.Job:  map 29% reduce 0%
      12/02/03 21:07:17 INFO mapreduce.Job:  map 35% reduce 0%
      12/02/03 21:07:30 INFO mapreduce.Job:  map 52% reduce 0%
      12/02/03 21:07:35 INFO mapreduce.Job:  map 58% reduce 0%
      12/02/03 21:07:37 INFO mapreduce.Job:  map 70% reduce 0%
      12/02/03 21:07:41 INFO mapreduce.Job:  map 76% reduce 0%
      12/02/03 21:07:43 INFO mapreduce.Job:  map 82% reduce 0%
      12/02/03 21:07:44 INFO mapreduce.Job:  map 88% reduce 0%
      12/02/03 21:07:47 INFO mapreduce.Job:  map 94% reduce 0%
      12/02/03 21:07:49 INFO mapreduce.Job:  map 100% reduce 0%
      12/02/03 21:07:53 INFO mapreduce.Job:  map 100% reduce 3%
      12/02/03 21:08:00 INFO mapreduce.Job:  map 100% reduce 6%
      12/02/03 21:08:06 INFO mapreduce.Job:  map 100% reduce 10%
      12/02/03 21:08:12 INFO mapreduce.Job:  map 100% reduce 13%
      12/02/03 21:08:18 INFO mapreduce.Job:  map 100% reduce 16%
      #killed AM with kill -9 here
      12/02/03 21:08:20 INFO ipc.Client: Retrying connect to server: HOST/IP:44223. Already tried 0 time(s).
      12/02/03 21:08:21 INFO ipc.Client: Retrying connect to server: HOST/IP:44223. Already tried 1 time(s).
      12/02/03 21:08:22 INFO ipc.Client: Retrying connect to server: HOST/IP:44223. Already tried 2 time(s).
      12/02/03 21:08:26 INFO mapreduce.Job:  map 64% reduce 16%
      #It never makes any more progress...
      
      1. syslog
        683 kB
        Robert Joseph Evans
      2. MAPREDUCE-3802-20120213.txt
        6 kB
        Vinod Kumar Vavilapalli
      3. MAPREDUCE-3802-20120213.txt
        6 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Hide
          Robert Joseph Evans added a comment -

          This is the syslog with debug enabled for the MR AM from 0.23 for the MRAM that livelocks.

          Show
          Robert Joseph Evans added a comment - This is the syslog with debug enabled for the MR AM from 0.23 for the MRAM that livelocks.
          Hide
          Robert Joseph Evans added a comment -

          From talking with others it looks like this may only happen on a single node cluster as well. The fact that the mapper % dropped a lot seems a bit odd, almost like it lost some mapper tasks as part of recovery.

          Show
          Robert Joseph Evans added a comment - From talking with others it looks like this may only happen on a single node cluster as well. The fact that the mapper % dropped a lot seems a bit odd, almost like it lost some mapper tasks as part of recovery.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          This is inline with I always used to see while working on sub-tickets at MAPREDUCE-2692 but on my localhost. Contrary to this, Karam never saw this on a cluster. So hopefully and before any further testing reveals otherwise, I treat this as a non-blocker.

          Show
          Vinod Kumar Vavilapalli added a comment - This is inline with I always used to see while working on sub-tickets at MAPREDUCE-2692 but on my localhost. Contrary to this, Karam never saw this on a cluster. So hopefully and before any further testing reveals otherwise, I treat this as a non-blocker.
          Hide
          Robert Joseph Evans added a comment -

          I just found the following in the log

          2012-02-03 21:08:24,401 FATAL [AsyncDispatcher event handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread. Exiting..
          java.lang.NullPointerException
                  at org.apache.hadoop.mapreduce.v2.app.recover.RecoveryService$InterceptingEventHandler.sendAssignedEvent(RecoveryService.java:437)
                  at org.apache.hadoop.mapreduce.v2.app.recover.RecoveryService$InterceptingEventHandler.handle(RecoveryService.java:336)
                  at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$RequestContainerTransition.transition(TaskAttemptImpl.java:1082)
                  at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$RequestContainerTransition.transition(TaskAttemptImpl.java:1)
                  at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
                  at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
                  at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
                  at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
                  at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:919)
                  at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1)
                  at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:871)
                  at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:1)
                  at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:125)
                  at org.apache.hadoop.mapreduce.v2.app.recover.RecoveryService$RecoveryDispatcher.realDispatch(RecoveryService.java:291)
                  at org.apache.hadoop.mapreduce.v2.app.recover.RecoveryService$RecoveryDispatcher.dispatch(RecoveryService.java:287)
                  at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:82)
                  at java.lang.Thread.run(Thread.java:662)
          

          It looks like for some reason someone is sending a CONTAINER_REQ event without an attempt ID.

          Show
          Robert Joseph Evans added a comment - I just found the following in the log 2012-02-03 21:08:24,401 FATAL [AsyncDispatcher event handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread. Exiting.. java.lang.NullPointerException at org.apache.hadoop.mapreduce.v2.app.recover.RecoveryService$InterceptingEventHandler.sendAssignedEvent(RecoveryService.java:437) at org.apache.hadoop.mapreduce.v2.app.recover.RecoveryService$InterceptingEventHandler.handle(RecoveryService.java:336) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$RequestContainerTransition.transition(TaskAttemptImpl.java:1082) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$RequestContainerTransition.transition(TaskAttemptImpl.java:1) at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357) at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298) at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43) at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:919) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:871) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:1) at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:125) at org.apache.hadoop.mapreduce.v2.app.recover.RecoveryService$RecoveryDispatcher.realDispatch(RecoveryService.java:291) at org.apache.hadoop.mapreduce.v2.app.recover.RecoveryService$RecoveryDispatcher.dispatch(RecoveryService.java:287) at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:82) at java.lang.Thread.run(Thread.java:662) It looks like for some reason someone is sending a CONTAINER_REQ event without an attempt ID.
          Hide
          Robert Joseph Evans added a comment -

          Sorry I traced that wrong. It actually looks like the Task does not have an attempt info for that task attempt.

          Show
          Robert Joseph Evans added a comment - Sorry I traced that wrong. It actually looks like the Task does not have an attempt info for that task attempt.
          Hide
          Mahadev konar added a comment -

          @Bobby,
          So this is an issue with the events being logged in the job history file?

          Show
          Mahadev konar added a comment - @Bobby, So this is an issue with the events being logged in the job history file?
          Hide
          Robert Joseph Evans added a comment -

          That appears to be the case. We are getting an NPE which is caused by calling RecoverService.getTaskAttemptInfo() and getting a null back. RecoveryService.getTaskAttemptInfo() first gets a task info, and then gets a task attempt info from inside that task. It looks like the task info is parsed and populated just fine, but the task attempt info is not. That seems to be caused by no TaskAttemptStarted events being put in the history log at all during the recovery process. This also seems like no MapAttemptFinishedEvents, ReduceAttemp0tFinishedEvents, TaskAttemptFailedEvents nor TaskAttemptFinishedEvents are in the log either, or we would get null pointer exceptions while parsing them too.

          Show
          Robert Joseph Evans added a comment - That appears to be the case. We are getting an NPE which is caused by calling RecoverService.getTaskAttemptInfo() and getting a null back. RecoveryService.getTaskAttemptInfo() first gets a task info, and then gets a task attempt info from inside that task. It looks like the task info is parsed and populated just fine, but the task attempt info is not. That seems to be caused by no TaskAttemptStarted events being put in the history log at all during the recovery process. This also seems like no MapAttemptFinishedEvents, ReduceAttemp0tFinishedEvents, TaskAttemptFailedEvents nor TaskAttemptFinishedEvents are in the log either, or we would get null pointer exceptions while parsing them too.
          Hide
          Robert Joseph Evans added a comment -

          I have been looking for differences in the jhist files between the initial kill, and the second one, and I cannot find much as far as the content is concerned. So I think it might have something to do with the order of the events.

          In the second jhist file there are two AMStarted events. I assume that is to allow for the history server to show how many have been started.

          The mapFinishTime of all the new MAP_ATTEMPT_FINISH_EVENTS is 0. The finishTime for these events are all showing the same finish time which I think is when the attempt was recovered, not the original event finish time. The state of the event changed from "map" (which seems like a bug) to "SUCCEEDED" which looks more correct to me. And finally the clockSplits counters are all 0 in the new one as well.

          Similarly the TASK_FINISH_EVENTS have the finish time of the recovery, not the actual finish time.

          I could not find anything else that is significantly different.

          Show
          Robert Joseph Evans added a comment - I have been looking for differences in the jhist files between the initial kill, and the second one, and I cannot find much as far as the content is concerned. So I think it might have something to do with the order of the events. In the second jhist file there are two AMStarted events. I assume that is to allow for the history server to show how many have been started. The mapFinishTime of all the new MAP_ATTEMPT_FINISH_EVENTS is 0. The finishTime for these events are all showing the same finish time which I think is when the attempt was recovered, not the original event finish time. The state of the event changed from "map" (which seems like a bug) to "SUCCEEDED" which looks more correct to me. And finally the clockSplits counters are all 0 in the new one as well. Similarly the TASK_FINISH_EVENTS have the finish time of the recovery, not the actual finish time. I could not find anything else that is significantly different.
          Hide
          Robert Joseph Evans added a comment -

          OK I found the issue, sort of, and it has nothing to do with order. The issue is with the name of the task attempt. If the task was completed by the first AM and recovered by the second AM the name of the task attempt in the jhist file will look like attempt_1328637230353_0001_m_000000_0, but the Recovery Service is trying to recover a task with attempt id attempt_1328637230353_0001_m_000000_1000, which appears to be the format for attempts that completed successfully with the second AM.

          I need to understand a little bit better how these names are determined, and where they are set, so I can determine how to fix the issue. I don't see how this could only be a problem for a single node cluster.

          Show
          Robert Joseph Evans added a comment - OK I found the issue, sort of, and it has nothing to do with order. The issue is with the name of the task attempt. If the task was completed by the first AM and recovered by the second AM the name of the task attempt in the jhist file will look like attempt_1328637230353_0001_m_000000_0, but the Recovery Service is trying to recover a task with attempt id attempt_1328637230353_0001_m_000000_1000, which appears to be the format for attempts that completed successfully with the second AM. I need to understand a little bit better how these names are determined, and where they are set, so I can determine how to fix the issue. I don't see how this could only be a problem for a single node cluster.
          Hide
          Sharad Agarwal added a comment -

          need to understand a little bit better how these names are determined

          The task attemptIds are unique across all the generations of AM. This is to avoid any remote task attempt from previous generation of AM joining the current AM. The assumption is there won't be more than 1000 attempts of a task in AM run. The suffix part of task attemptId is determined as follows:
          _(AMGeneration-1)*1000. For first AM it will start from 0. For second it will start from 1000, for third from 2000 ..

          Show
          Sharad Agarwal added a comment - need to understand a little bit better how these names are determined The task attemptIds are unique across all the generations of AM. This is to avoid any remote task attempt from previous generation of AM joining the current AM. The assumption is there won't be more than 1000 attempts of a task in AM run. The suffix part of task attemptId is determined as follows: _(AMGeneration-1)*1000. For first AM it will start from 0. For second it will start from 1000, for third from 2000 ..
          Hide
          Sharad Agarwal added a comment -

          The bug is in TaskImpl over here:

              //attempt ids are generated based on MR app startCount so that attempts
              //from previous lives don't overstep the current one.
              //this assumes that a task won't have more than 1000 attempts in its single 
              //life
              nextAttemptNumber = (startCount - 1) * 1000;
          

          The completed task could be from any earlier AM generation not just from the previous one. I am looking into the way to fix this.

          Show
          Sharad Agarwal added a comment - The bug is in TaskImpl over here: //attempt ids are generated based on MR app startCount so that attempts //from previous lives don't overstep the current one. // this assumes that a task won't have more than 1000 attempts in its single //life nextAttemptNumber = (startCount - 1) * 1000; The completed task could be from any earlier AM generation not just from the previous one. I am looking into the way to fix this.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          MAPREDUCE-3846 is a related ticket with same symptom but a different cause.

          I uploaded a patch there, where instead of generating AttemptIDs sequentially when recovering, I am using the previous generation attemptIDs first before moving onto generating new ones for this generation. I thought for a while and now it seems to me that that patch will automatically fix this issue also, except I don't have a test validating this.

          Robert/Sharad, can you please look at my patch there and see if that fixes this? Thanks.

          Show
          Vinod Kumar Vavilapalli added a comment - MAPREDUCE-3846 is a related ticket with same symptom but a different cause. I uploaded a patch there, where instead of generating AttemptIDs sequentially when recovering, I am using the previous generation attemptIDs first before moving onto generating new ones for this generation. I thought for a while and now it seems to me that that patch will automatically fix this issue also, except I don't have a test validating this. Robert/Sharad, can you please look at my patch there and see if that fixes this? Thanks.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Muy latest patch at MAPREDUCE-3846 is automatically fixing this (I think). I am adding a test here to validate/invalidate that that is the case.

          Show
          Vinod Kumar Vavilapalli added a comment - Muy latest patch at MAPREDUCE-3846 is automatically fixing this (I think). I am adding a test here to validate/invalidate that that is the case.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          This new test fails on the trunk (i.e. before MAPREDUCE-3846 goes in).

          Show
          Vinod Kumar Vavilapalli added a comment - This new test fails on the trunk (i.e. before MAPREDUCE-3846 goes in).
          Hide
          Vinod Kumar Vavilapalli added a comment -

          The patch can go in only after MAPREDUCE-3846.

          Show
          Vinod Kumar Vavilapalli added a comment - The patch can go in only after MAPREDUCE-3846 .
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12514406/MAPREDUCE-3802-20120213.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.mapreduce.v2.app.TestRecovery

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1848//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1848//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514406/MAPREDUCE-3802-20120213.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapreduce.v2.app.TestRecovery +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1848//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1848//console This message is automatically generated.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          MAPREDUCE-3846 is in, reattaching the same patch.

          Show
          Vinod Kumar Vavilapalli added a comment - MAPREDUCE-3846 is in, reattaching the same patch.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12514424/MAPREDUCE-3802-20120213.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1849//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1849//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514424/MAPREDUCE-3802-20120213.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1849//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1849//console This message is automatically generated.
          Hide
          Robert Joseph Evans added a comment -

          +1 you are correct I manually verified that 0.23.2 does not show this problem any more, and so I assume that it is MAPREDUCE-3846 that fixed it.

          Show
          Robert Joseph Evans added a comment - +1 you are correct I manually verified that 0.23.2 does not show this problem any more, and so I assume that it is MAPREDUCE-3846 that fixed it.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Thanks for the review/verification, Robert!

          I just committed this to trunk, 0.23 and 0.23.1.

          Show
          Vinod Kumar Vavilapalli added a comment - Thanks for the review/verification, Robert! I just committed this to trunk, 0.23 and 0.23.1.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #537 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/537/)
          MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv)
          svn merge --ignore-ancestry -c 1244178 ../../trunk/ (Revision 1244180)

          Result = SUCCESS
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244180
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #537 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/537/ ) MAPREDUCE-3802 . Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 . (vinodkv) svn merge --ignore-ancestry -c 1244178 ../../trunk/ (Revision 1244180) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244180 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #549 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/549/)
          MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv)
          svn merge --ignore-ancestry -c 1244178 ../../trunk/ (Revision 1244180)

          Result = SUCCESS
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244180
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #549 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/549/ ) MAPREDUCE-3802 . Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 . (vinodkv) svn merge --ignore-ancestry -c 1244178 ../../trunk/ (Revision 1244180) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244180 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1800 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1800/)
          MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv) (Revision 1244178)

          Result = SUCCESS
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244178
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1800 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1800/ ) MAPREDUCE-3802 . Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 . (vinodkv) (Revision 1244178) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244178 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1726 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1726/)
          MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv) (Revision 1244178)

          Result = SUCCESS
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244178
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1726 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1726/ ) MAPREDUCE-3802 . Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 . (vinodkv) (Revision 1244178) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244178 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #553 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/553/)
          MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv)
          svn merge --ignore-ancestry -c 1244178 ../../trunk/ (Revision 1244180)

          Result = ABORTED
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244180
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #553 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/553/ ) MAPREDUCE-3802 . Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 . (vinodkv) svn merge --ignore-ancestry -c 1244178 ../../trunk/ (Revision 1244180) Result = ABORTED vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244180 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1737 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1737/)
          MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv) (Revision 1244178)

          Result = ABORTED
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244178
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1737 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1737/ ) MAPREDUCE-3802 . Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 . (vinodkv) (Revision 1244178) Result = ABORTED vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244178 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #956 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/956/)
          MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv) (Revision 1244178)

          Result = FAILURE
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244178
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #956 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/956/ ) MAPREDUCE-3802 . Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 . (vinodkv) (Revision 1244178) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244178 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #169 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/169/)
          MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv)
          svn merge --ignore-ancestry -c 1244178 ../../trunk/ (Revision 1244180)

          Result = FAILURE
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244180
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #169 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/169/ ) MAPREDUCE-3802 . Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 . (vinodkv) svn merge --ignore-ancestry -c 1244178 ../../trunk/ (Revision 1244180) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244180 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #197 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/197/)
          MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv)
          svn merge --ignore-ancestry -c 1244178 ../../trunk/ (Revision 1244180)

          Result = FAILURE
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244180
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #197 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/197/ ) MAPREDUCE-3802 . Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 . (vinodkv) svn merge --ignore-ancestry -c 1244178 ../../trunk/ (Revision 1244180) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244180 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #991 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/991/)
          MAPREDUCE-3802. Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. (vinodkv) (Revision 1244178)

          Result = SUCCESS
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244178
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #991 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/991/ ) MAPREDUCE-3802 . Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846 . (vinodkv) (Revision 1244178) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244178 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Robert Joseph Evans
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development