Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4637

Killing an unassigned task attempt causes the job to fail

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.3-alpha, 0.23.7
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states.

      Description

      Attempting to kill a task attempt that has been scheduled but is not running causes an invalid state transition and the AM to stop with an error.

      1. MAPREDUCE-4637-trunk-v6.patch
        7 kB
        Mayank Bansal
      2. MAPREDUCE-4637-trunk-v5.patch
        9 kB
        Mayank Bansal
      3. MAPREDUCE-4637-trunk-v4.patch
        10 kB
        Mayank Bansal
      4. MAPREDUCE-4637-trunk-v3.patch
        9 kB
        Mayank Bansal
      5. MapReduce.png
        693 kB
        Tom White
      6. MAPREDUCE-4637-trunk-v2.patch
        4 kB
        Mayank Bansal
      7. MAPREDUCE-4637-trunk.patch
        4 kB
        Mayank Bansal

        Activity

        Tom White created issue -
        Hide
        Tom White added a comment -

        Here are the relevant lines from the AM's log when a task attempt (attempt_1337372605417_0946_m_000095_0) was killed manually from the CLI:

        2012-05-24 11:45:49,328 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1337372605417_0946_m_000095_0 TaskAttempt Transitioned from NEW to UNASSIGNED
        2012-05-24 12:01:55,072 INFO [IPC Server handler 0 on 53369] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Kill task attempt received from client attempt_1337372605417_0946_m_000095_0
        2012-05-24 12:01:55,073 ERROR [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Can't handle this event at current state for attempt_1337372605417_0946_m_000095_0
        org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: TA_DIAGNOSTICS_UPDATE at UNASSIGNED
                at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
                at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
                at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
                at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:941)
                at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:133)
                at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:912)
                at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:904)
                at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:125)
                at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:74)
                at java.lang.Thread.run(Thread.java:662)
        2012-05-24 12:01:55,076 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1337372605417_0946_m_000095_0 TaskAttempt Transitioned from UNASSIGNED to KILLED
        2012-05-24 12:01:55,078 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1337372605417_0946Job Transitioned from RUNNING to ERROR
        2012-05-24 12:01:55,078 INFO [Thread-46] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Processing the event EventType: CONTAINER_DEALLOCATE
        

        The bugs seems to be in MRClientService.MRClientProtocolHandler#killTaskAttempt() where the handler is sent a TaskAttemptDiagnosticsUpdateEvent. However, in the UNASSIGNED state a TaskAttempt cannot handle this event, so it fails with an error. The simplest fix would be to allow a TaskAttemptDiagnosticsUpdateEvent in the UNASSIGNED state. The same bug occurs with the fail task command.

        This bug was found by Wing Yew Poon.

        Show
        Tom White added a comment - Here are the relevant lines from the AM's log when a task attempt (attempt_1337372605417_0946_m_000095_0) was killed manually from the CLI: 2012-05-24 11:45:49,328 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1337372605417_0946_m_000095_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2012-05-24 12:01:55,072 INFO [IPC Server handler 0 on 53369] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Kill task attempt received from client attempt_1337372605417_0946_m_000095_0 2012-05-24 12:01:55,073 ERROR [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Can't handle this event at current state for attempt_1337372605417_0946_m_000095_0 org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: TA_DIAGNOSTICS_UPDATE at UNASSIGNED at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301) at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43) at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:941) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:133) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:912) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:904) at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:125) at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:74) at java.lang.Thread.run(Thread.java:662) 2012-05-24 12:01:55,076 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1337372605417_0946_m_000095_0 TaskAttempt Transitioned from UNASSIGNED to KILLED 2012-05-24 12:01:55,078 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1337372605417_0946Job Transitioned from RUNNING to ERROR 2012-05-24 12:01:55,078 INFO [Thread-46] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Processing the event EventType: CONTAINER_DEALLOCATE The bugs seems to be in MRClientService.MRClientProtocolHandler#killTaskAttempt() where the handler is sent a TaskAttemptDiagnosticsUpdateEvent. However, in the UNASSIGNED state a TaskAttempt cannot handle this event, so it fails with an error. The simplest fix would be to allow a TaskAttemptDiagnosticsUpdateEvent in the UNASSIGNED state. The same bug occurs with the fail task command. This bug was found by Wing Yew Poon.
        Hide
        Mayank Bansal added a comment -

        Attaching the patch.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Attaching the patch. Thanks, Mayank
        Mayank Bansal made changes -
        Field Original Value New Value
        Attachment MAPREDUCE-4637-trunk.patch [ 12543899 ]
        Mayank Bansal made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12543899/MAPREDUCE-4637-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2817//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2817//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543899/MAPREDUCE-4637-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2817//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2817//console This message is automatically generated.
        Hide
        Tom White added a comment -

        Thanks for the patch Mayank. It looks good to me, and when I ran the test without the fix it failed. The only thing I think needs changing is the method name for the test, which should be lowerCamelCase.

        Show
        Tom White added a comment - Thanks for the patch Mayank. It looks good to me, and when I ran the test without the fix it failed. The only thing I think needs changing is the method name for the test, which should be lowerCamelCase.
        Tom White made changes -
        Assignee Mayank Bansal [ mayank_bansal ]
        Hide
        Mayank Bansal added a comment -

        Thanks Tom for the review. I updated the patch with your comments.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Thanks Tom for the review. I updated the patch with your comments. Thanks, Mayank
        Mayank Bansal made changes -
        Attachment MAPREDUCE-4637-trunk-v2.patch [ 12543911 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12543911/MAPREDUCE-4637-trunk-v2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2820//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2820//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543911/MAPREDUCE-4637-trunk-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2820//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2820//console This message is automatically generated.
        Hide
        Arun C Murthy added a comment -

        Another option is to validate that a kill is only sent to RUNNING task-attempts. That seems a better option, no?

        Show
        Arun C Murthy added a comment - Another option is to validate that a kill is only sent to RUNNING task-attempts. That seems a better option, no?
        Hide
        Mayank Bansal added a comment -

        I think we should kill all the tasks irrespective of whatever state they are in. Isn't it?

        Show
        Mayank Bansal added a comment - I think we should kill all the tasks irrespective of whatever state they are in. Isn't it?
        Hide
        Tom White added a comment -

        Looking at the state transition diagrams (attached) it's possible to kill a job, task or task attempt in any of their pre-running states: NEW, INITED/SCHEDULED/(UN)ASSIGNED. I don't see a reason to change that since users may have a reason to kill a task attempt before it is running. The bug here is that the unassigned state of the task attempt does not have a TA_DIAGNOSTICS_UPDATE transition which causes the kill to fail.

        I just noticed that job has a JOB_DIAGNOSTIC_UPDATE transition from both the NEW and INITED states, so perhaps we should extend the patch to add a TA_DIAGNOSTICS_UPDATE from the NEW state of task attempt too for completeness.

        Show
        Tom White added a comment - Looking at the state transition diagrams (attached) it's possible to kill a job, task or task attempt in any of their pre-running states: NEW, INITED/SCHEDULED/(UN)ASSIGNED. I don't see a reason to change that since users may have a reason to kill a task attempt before it is running. The bug here is that the unassigned state of the task attempt does not have a TA_DIAGNOSTICS_UPDATE transition which causes the kill to fail. I just noticed that job has a JOB_DIAGNOSTIC_UPDATE transition from both the NEW and INITED states, so perhaps we should extend the patch to add a TA_DIAGNOSTICS_UPDATE from the NEW state of task attempt too for completeness.
        Tom White made changes -
        Attachment MapReduce.png [ 12544026 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544026/MapReduce.png
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2824//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544026/MapReduce.png against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2824//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        Thanks Tom for sending the state diagram. It is very helpful.

        I think its a good idea for adding TA_DIAGNOSTICS_UPDATE from the NEW state.

        Arun,

        Do you think its reasonable?

        I can add this state in this patch itself or you want me to open another jira for that?

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Thanks Tom for sending the state diagram. It is very helpful. I think its a good idea for adding TA_DIAGNOSTICS_UPDATE from the NEW state. Arun, Do you think its reasonable? I can add this state in this patch itself or you want me to open another jira for that? Thanks, Mayank
        Hide
        Tom White added a comment -

        BTW you can generate the state diagram by following the instructions in YARN's README.

        Show
        Tom White added a comment - BTW you can generate the state diagram by following the instructions in YARN's README.
        Hide
        Mayank Bansal added a comment -

        Based on Tom's suggestion extending patch for the new state as well.

        Arun,

        Did you get a chance to take a look?

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Based on Tom's suggestion extending patch for the new state as well. Arun, Did you get a chance to take a look? Thanks, Mayank
        Mayank Bansal made changes -
        Attachment MAPREDUCE-4637-trunk-v3.patch [ 12544522 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544522/MAPREDUCE-4637-trunk-v3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app:

        org.apache.hadoop.mapreduce.v2.app.TestMRClientService

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2837//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2837//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544522/MAPREDUCE-4637-trunk-v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: org.apache.hadoop.mapreduce.v2.app.TestMRClientService +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2837//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2837//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        Fixing the test.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Fixing the test. Thanks, Mayank
        Mayank Bansal made changes -
        Attachment MAPREDUCE-4637-trunk-v4.patch [ 12544559 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544559/MAPREDUCE-4637-trunk-v4.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2840//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2840//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544559/MAPREDUCE-4637-trunk-v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2840//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2840//console This message is automatically generated.
        Hide
        Siddharth Seth added a comment -

        I don't see a reason to change that since users may have a reason to kill a task attempt before it is running. The bug here is that the unassigned state of the task attempt does not have a TA_DIAGNOSTICS_UPDATE transition which causes the kill to fail.

        A kill seems to be a valid operation before an attempt starts. At a later point, maybe we should include diagnostic messages along with the actual event instead of 2 independent events.

        The patch needs some minor changes though. There's an extra Diagnostic Event being generated from TaskImpl - which isn't required. The unit tests may be mocking more than they should, but I don't see that as being an issue.

        Show
        Siddharth Seth added a comment - I don't see a reason to change that since users may have a reason to kill a task attempt before it is running. The bug here is that the unassigned state of the task attempt does not have a TA_DIAGNOSTICS_UPDATE transition which causes the kill to fail. A kill seems to be a valid operation before an attempt starts. At a later point, maybe we should include diagnostic messages along with the actual event instead of 2 independent events. The patch needs some minor changes though. There's an extra Diagnostic Event being generated from TaskImpl - which isn't required. The unit tests may be mocking more than they should, but I don't see that as being an issue.
        Siddharth Seth made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Mayank Bansal added a comment -

        Thanks sid for the review, I updated the patch with your comments. I rebased the patch as well with the latest source.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Thanks sid for the review, I updated the patch with your comments. I rebased the patch as well with the latest source. Thanks, Mayank
        Mayank Bansal made changes -
        Attachment MAPREDUCE-4637-trunk-v5.patch [ 12550118 ]
        Mayank Bansal made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12550118/MAPREDUCE-4637-trunk-v5.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app:

        org.apache.hadoop.mapreduce.v2.app.TestMRClientService

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2949//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2949//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12550118/MAPREDUCE-4637-trunk-v5.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: org.apache.hadoop.mapreduce.v2.app.TestMRClientService +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2949//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2949//console This message is automatically generated.
        Mayank Bansal made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Mayank Bansal added a comment -

        Fixing Test.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Fixing Test. Thanks, Mayank
        Mayank Bansal made changes -
        Attachment MAPREDUCE-4637-trunk-v6.patch [ 12550133 ]
        Mayank Bansal made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12550133/MAPREDUCE-4637-trunk-v6.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2950//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2950//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12550133/MAPREDUCE-4637-trunk-v6.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2950//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2950//console This message is automatically generated.
        Hide
        Siddharth Seth added a comment -

        +1. Committing this. Thanks Mayank.

        Show
        Siddharth Seth added a comment - +1. Committing this. Thanks Mayank.
        Hide
        Siddharth Seth added a comment -

        Committed to trunk and branch-2.

        Show
        Siddharth Seth added a comment - Committed to trunk and branch-2.
        Siddharth Seth made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Release Note Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states.
        Fix Version/s 2.0.3-alpha [ 12323275 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #2917 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2917/)
        MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #2917 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2917/ ) MAPREDUCE-4637 . Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #13 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/13/)
        MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474)

        Result = FAILURE
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #13 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/13/ ) MAPREDUCE-4637 . Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1205 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1205/)
        MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474)

        Result = FAILURE
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1205 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1205/ ) MAPREDUCE-4637 . Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1235 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1235/)
        MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1235 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1235/ ) MAPREDUCE-4637 . Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Contributed by Mayank Bansal) (Revision 1401474) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1401474 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Thomas Graves made changes -
        Fix Version/s 0.23.7 [ 12323954 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #518 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/518/)
        MAPREDUCE-4637. Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Mayank Bansal via tgraves) (Revision 1443243)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443243
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #518 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/518/ ) MAPREDUCE-4637 . Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. (Mayank Bansal via tgraves) (Revision 1443243) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443243 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Mayank Bansal
            Reporter:
            Tom White
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development