Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4637

Killing an unassigned task attempt causes the job to fail

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.3-alpha, 0.23.7
    • Component/s: mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states.
    • Target Version/s:

      Description

      Attempting to kill a task attempt that has been scheduled but is not running causes an invalid state transition and the AM to stop with an error.

      1. MapReduce.png
        693 kB
        Tom White
      2. MAPREDUCE-4637-trunk.patch
        4 kB
        Mayank Bansal
      3. MAPREDUCE-4637-trunk-v2.patch
        4 kB
        Mayank Bansal
      4. MAPREDUCE-4637-trunk-v3.patch
        9 kB
        Mayank Bansal
      5. MAPREDUCE-4637-trunk-v4.patch
        10 kB
        Mayank Bansal
      6. MAPREDUCE-4637-trunk-v5.patch
        9 kB
        Mayank Bansal
      7. MAPREDUCE-4637-trunk-v6.patch
        7 kB
        Mayank Bansal

        Activity

        Hide
        Tom White added a comment -

        Here are the relevant lines from the AM's log when a task attempt (attempt_1337372605417_0946_m_000095_0) was killed manually from the CLI:

        2012-05-24 11:45:49,328 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1337372605417_0946_m_000095_0 TaskAttempt Transitioned from NEW to UNASSIGNED
        2012-05-24 12:01:55,072 INFO [IPC Server handler 0 on 53369] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Kill task attempt received from client attempt_1337372605417_0946_m_000095_0
        2012-05-24 12:01:55,073 ERROR [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Can't handle this event at current state for attempt_1337372605417_0946_m_000095_0
        org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: TA_DIAGNOSTICS_UPDATE at UNASSIGNED
                at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
                at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
                at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
                at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:941)
                at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:133)
                at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:912)
                at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:904)
                at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:125)
                at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:74)
                at java.lang.Thread.run(Thread.java:662)
        2012-05-24 12:01:55,076 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1337372605417_0946_m_000095_0 TaskAttempt Transitioned from UNASSIGNED to KILLED
        2012-05-24 12:01:55,078 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1337372605417_0946Job Transitioned from RUNNING to ERROR
        2012-05-24 12:01:55,078 INFO [Thread-46] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Processing the event EventType: CONTAINER_DEALLOCATE
        

        The bugs seems to be in MRClientService.MRClientProtocolHandler#killTaskAttempt() where the handler is sent a TaskAttemptDiagnosticsUpdateEvent. However, in the UNASSIGNED state a TaskAttempt cannot handle this event, so it fails with an error. The simplest fix would be to allow a TaskAttemptDiagnosticsUpdateEvent in the UNASSIGNED state. The same bug occurs with the fail task command.

        This bug was found by Wing Yew Poon.

        Show
        Tom White added a comment - Here are the relevant lines from the AM's log when a task attempt (attempt_1337372605417_0946_m_000095_0) was killed manually from the CLI: 2012-05-24 11:45:49,328 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1337372605417_0946_m_000095_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2012-05-24 12:01:55,072 INFO [IPC Server handler 0 on 53369] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Kill task attempt received from client attempt_1337372605417_0946_m_000095_0 2012-05-24 12:01:55,073 ERROR [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Can't handle this event at current state for attempt_1337372605417_0946_m_000095_0 org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: TA_DIAGNOSTICS_UPDATE at UNASSIGNED at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301) at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43) at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:941) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:133) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:912) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:904) at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:125) at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:74) at java.lang.Thread.run(Thread.java:662) 2012-05-24 12:01:55,076 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1337372605417_0946_m_000095_0 TaskAttempt Transitioned from UNASSIGNED to KILLED 2012-05-24 12:01:55,078 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1337372605417_0946Job Transitioned from RUNNING to ERROR 2012-05-24 12:01:55,078 INFO [Thread-46] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Processing the event EventType: CONTAINER_DEALLOCATE The bugs seems to be in MRClientService.MRClientProtocolHandler#killTaskAttempt() where the handler is sent a TaskAttemptDiagnosticsUpdateEvent. However, in the UNASSIGNED state a TaskAttempt cannot handle this event, so it fails with an error. The simplest fix would be to allow a TaskAttemptDiagnosticsUpdateEvent in the UNASSIGNED state. The same bug occurs with the fail task command. This bug was found by Wing Yew Poon.
        Hide
        Mayank Bansal added a comment -

        Attaching the patch.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Attaching the patch. Thanks, Mayank
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12543899/MAPREDUCE-4637-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2817//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2817//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543899/MAPREDUCE-4637-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2817//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2817//console This message is automatically generated.
        Hide
        Tom White added a comment -

        Thanks for the patch Mayank. It looks good to me, and when I ran the test without the fix it failed. The only thing I think needs changing is the method name for the test, which should be lowerCamelCase.

        Show
        Tom White added a comment - Thanks for the patch Mayank. It looks good to me, and when I ran the test without the fix it failed. The only thing I think needs changing is the method name for the test, which should be lowerCamelCase.
        Hide
        Mayank Bansal added a comment -

        Thanks Tom for the review. I updated the patch with your comments.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Thanks Tom for the review. I updated the patch with your comments. Thanks, Mayank
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12543911/MAPREDUCE-4637-trunk-v2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2820//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2820//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543911/MAPREDUCE-4637-trunk-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2820//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2820//console This message is automatically generated.
        Hide
        Arun C Murthy added a comment -

        Another option is to validate that a kill is only sent to RUNNING task-attempts. That seems a better option, no?

        Show
        Arun C Murthy added a comment - Another option is to validate that a kill is only sent to RUNNING task-attempts. That seems a better option, no?
        Hide
        Mayank Bansal added a comment -

        I think we should kill all the tasks irrespective of whatever state they are in. Isn't it?

        Show
        Mayank Bansal added a comment - I think we should kill all the tasks irrespective of whatever state they are in. Isn't it?
        Hide
        Tom White added a comment -

        Looking at the state transition diagrams (attached) it's possible to kill a job, task or task attempt in any of their pre-running states: NEW, INITED/SCHEDULED/(UN)ASSIGNED. I don't see a reason to change that since users may have a reason to kill a task attempt before it is running. The bug here is that the unassigned state of the task attempt does not have a TA_DIAGNOSTICS_UPDATE transition which causes the kill to fail.

        I just noticed that job has a JOB_DIAGNOSTIC_UPDATE transition from both the NEW and INITED states, so perhaps we should extend the patch to add a TA_DIAGNOSTICS_UPDATE from the NEW state of task attempt too for completeness.

        Show
        Tom White added a comment - Looking at the state transition diagrams (attached) it's possible to kill a job, task or task attempt in any of their pre-running states: NEW, INITED/SCHEDULED/(UN)ASSIGNED. I don't see a reason to change that since users may have a reason to kill a task attempt before it is running. The bug here is that the unassigned state of the task attempt does not have a TA_DIAGNOSTICS_UPDATE transition which causes the kill to fail. I just noticed that job has a JOB_DIAGNOSTIC_UPDATE transition from both the NEW and INITED states, so perhaps we should extend the patch to add a TA_DIAGNOSTICS_UPDATE from the NEW state of task attempt too for completeness.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544026/MapReduce.png
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2824//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544026/MapReduce.png against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2824//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        Thanks Tom for sending the state diagram. It is very helpful.

        I think its a good idea for adding TA_DIAGNOSTICS_UPDATE from the NEW state.

        Arun,

        Do you think its reasonable?

        I can add this state in this patch itself or you want me to open another jira for that?

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Thanks Tom for sending the state diagram. It is very helpful. I think its a good idea for adding TA_DIAGNOSTICS_UPDATE from the NEW state. Arun, Do you think its reasonable? I can add this state in this patch itself or you want me to open another jira for that? Thanks, Mayank
        Hide
        Tom White added a comment -

        BTW you can generate the state diagram by following the instructions in YARN's README.

        Show
        Tom White added a comment - BTW you can generate the state diagram by following the instructions in YARN's README.
        Hide
        Mayank Bansal added a comment -

        Based on Tom's suggestion extending patch for the new state as well.

        Arun,

        Did you get a chance to take a look?

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Based on Tom's suggestion extending patch for the new state as well. Arun, Did you get a chance to take a look? Thanks, Mayank
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544522/MAPREDUCE-4637-trunk-v3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app:

        org.apache.hadoop.mapreduce.v2.app.TestMRClientService

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2837//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2837//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544522/MAPREDUCE-4637-trunk-v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: org.apache.hadoop.mapreduce.v2.app.TestMRClientService +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2837//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2837//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        Fixing the test.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Fixing the test. Thanks, Mayank
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544559/MAPREDUCE-4637-trunk-v4.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2840//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2840//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544559/MAPREDUCE-4637-trunk-v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2840//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2840//console This message is automatically generated.
        Hide
        Siddharth Seth added a comment -

        I don't see a reason to change that since users may have a reason to kill a task attempt before it is running. The bug here is that the unassigned state of the task attempt does not have a TA_DIAGNOSTICS_UPDATE transition which causes the kill to fail.

        A kill seems to be a valid operation before an attempt starts. At a later point, maybe we should include diagnostic messages along with the actual event instead of 2 independent events.

        The patch needs some minor changes though. There's an extra Diagnostic Event being generated from TaskImpl - which isn't required. The unit tests may be mocking more than they should, but I don't see that as being an issue.

        Show
        Siddharth Seth added a comment - I don't see a reason to change that since users may have a reason to kill a task attempt before it is running. The bug here is that the unassigned state of the task attempt does not have a TA_DIAGNOSTICS_UPDATE transition which causes the kill to fail. A kill seems to be a valid operation before an attempt starts. At a later point, maybe we should include diagnostic messages along with the actual event instead of 2 independent events. The patch needs some minor changes though. There's an extra Diagnostic Event being generated from TaskImpl - which isn't required. The unit tests may be mocking more than they should, but I don't see that as being an issue.
        Hide
        Mayank Bansal added a comment -

        Thanks sid for the review, I updated the patch with your comments. I rebased the patch as well with the latest source.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Thanks sid for the review, I updated the patch with your comments. I rebased the patch as well with the latest source. Thanks, Mayank
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12550118/MAPREDUCE-4637-trunk-v5.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app:

        org.apache.hadoop.mapreduce.v2.app.TestMRClientService

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2949//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2949//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12550118/MAPREDUCE-4637-trunk-v5.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: org.apache.hadoop.mapreduce.v2.app.TestMRClientService +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2949//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2949//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        Fixing Test.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Fixing Test. Thanks, Mayank
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12550133/MAPREDUCE-4637-trunk-v6.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2950//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2950//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12550133/MAPREDUCE-4637-trunk-v6.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2950//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2950//console This message is automatically generated.
        Hide
        Siddharth Seth added a comment -

        +1. Committing this. Thanks Mayank.

        Show
        Siddharth Seth added a comment - +1. Committing this. Thanks Mayank.
        Hide
        Siddharth Seth added a comment -

        Committed to trunk and branch-2.

        Show
        Siddharth Seth added a comment - Committed to trunk and branch-2.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #2917 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2917/)
        MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #2917 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2917/ ) MAPREDUCE-4637 . Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #13 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/13/)
        MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474)

        Result = FAILURE
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #13 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/13/ ) MAPREDUCE-4637 . Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1205 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1205/)
        MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474)

        Result = FAILURE
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1205 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1205/ ) MAPREDUCE-4637 . Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1235 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1235/)
        MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1235 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1235/ ) MAPREDUCE-4637 . Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #518 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/518/)
        MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Mayank Bansal via tgraves) (Revision 1443243)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443243
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #518 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/518/ ) MAPREDUCE-4637 . Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Mayank Bansal via tgraves) (Revision 1443243) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443243 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java

          People

          • Assignee:
            Mayank Bansal
            Reporter:
            Tom White
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development