Hadoop YARN
  1. Hadoop YARN
  2. YARN-37

TestRMAppTransitions.testAppSubmittedKilled passes for the wrong reason

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.3
    • Fix Version/s: 2.0.2-alpha
    • Component/s: resourcemanager
    • Labels:
      None

      Description

      TestRMAppTransitions#testAppSubmittedKilled causes an invalid event exception but the test doesn't catch the error since the final app state is still killed. Killed for the wrong reason, but the final state is the same.

      1. YARN-37-trunk-v4.patch
        12 kB
        Mayank Bansal
      2. MAPREDUCE-4455-YARN-trunk-v3.patch
        11 kB
        Mayank Bansal
      3. MAPREDUCE-4455-YARN-trunk-v2.patch
        4 kB
        Mayank Bansal
      4. MAPREDUCE-4455-trunk-v2.patch
        4 kB
        Mayank Bansal
      5. MAPREDUCE-4455-trunk-v1.patch
        4 kB
        Mayank Bansal

        Activity

        Jason Lowe created issue -
        Hide
        Jason Lowe added a comment -

        Output from the test:

        2012-07-18 19:30:15,528 INFO  [main] rmapp.TestRMAppTransitions (TestRMAppTransitions.java:testAppSubmittedKill(363)) - --- START: testAppSubmittedKill---
        2012-07-18 19:30:15,768 INFO  [main] resourcetracker.InlineDispatcher (InlineDispatcher.java:dispatch(38)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: START
        2012-07-18 19:30:15,768 INFO  [main] rmapp.RMAppImpl (RMAppImpl.java:handle(499)) - application_1342639815540_0001 State change from NEW to SUBMITTED
        2012-07-18 19:30:15,769 INFO  [main] rmapp.RMAppImpl (RMAppImpl.java:handle(499)) - application_1342639815540_0001 State change from SUBMITTED to ACCEPTED
        2012-07-18 19:30:15,769 INFO  [main] resourcetracker.InlineDispatcher (InlineDispatcher.java:dispatch(38)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: KILL
        2012-07-18 19:30:15,770 INFO  [main] resourcetracker.InlineDispatcher (InlineDispatcher.java:dispatch(38)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppFailedAttemptEvent.EventType: ATTEMPT_KILLED
        2012-07-18 19:30:15,773 ERROR [main] rmapp.RMAppImpl (RMAppImpl.java:handle(494)) - Can't handle this event at current state
        org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: ATTEMPT_KILLED at ACCEPTED
                at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
                at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
                at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:492)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:70)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions$TestApplicationEventDispatcher.handle(TestRMAppTransitions.java:103)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions$TestApplicationEventDispatcher.handle(TestRMAppTransitions.java:89)
                at org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.InlineDispatcher.dispatch(InlineDispatcher.java:43)
                at org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.InlineDispatcher$TestEventHandler.handle(InlineDispatcher.java:33)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$BaseFinalTransition.transition(RMAppAttemptImpl.java:701)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$BaseFinalTransition.transition(RMAppAttemptImpl.java:651)
                at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
                at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
                at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
                at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:485)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:80)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions$TestApplicationAttemptEventDispatcher.handle(TestRMAppTransitions.java:79)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions$TestApplicationAttemptEventDispatcher.handle(TestRMAppTransitions.java:65)
                at org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.InlineDispatcher.dispatch(InlineDispatcher.java:43)
                at org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.InlineDispatcher$TestEventHandler.handle(InlineDispatcher.java:33)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl$KillAppAndAttemptTransition.transition(RMAppImpl.java:563)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl$KillAppAndAttemptTransition.transition(RMAppImpl.java:559)
                at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
                at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
                at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
                at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:492)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:70)
                at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions.testAppSubmittedKill(TestRMAppTransitions.java:369)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                at java.lang.reflect.Method.invoke(Method.java:597)
                at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
                at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
                at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
                at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
                at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
                at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
                at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
                at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
                at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
                at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
                at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
                at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
                at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
                at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
                at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:228)
                at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
                at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                at java.lang.reflect.Method.invoke(Method.java:597)
                at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
                at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
                at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
                at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
                at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
        

        It would be nice if the state machine transition tests would fail on any invalid state transition encountered during the tests so these can be caught more proactively in the future.

        Show
        Jason Lowe added a comment - Output from the test: 2012-07-18 19:30:15,528 INFO [main] rmapp.TestRMAppTransitions (TestRMAppTransitions.java:testAppSubmittedKill(363)) - --- START: testAppSubmittedKill--- 2012-07-18 19:30:15,768 INFO [main] resourcetracker.InlineDispatcher (InlineDispatcher.java:dispatch(38)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: START 2012-07-18 19:30:15,768 INFO [main] rmapp.RMAppImpl (RMAppImpl.java:handle(499)) - application_1342639815540_0001 State change from NEW to SUBMITTED 2012-07-18 19:30:15,769 INFO [main] rmapp.RMAppImpl (RMAppImpl.java:handle(499)) - application_1342639815540_0001 State change from SUBMITTED to ACCEPTED 2012-07-18 19:30:15,769 INFO [main] resourcetracker.InlineDispatcher (InlineDispatcher.java:dispatch(38)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEvent.EventType: KILL 2012-07-18 19:30:15,770 INFO [main] resourcetracker.InlineDispatcher (InlineDispatcher.java:dispatch(38)) - Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppFailedAttemptEvent.EventType: ATTEMPT_KILLED 2012-07-18 19:30:15,773 ERROR [main] rmapp.RMAppImpl (RMAppImpl.java:handle(494)) - Can't handle this event at current state org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: ATTEMPT_KILLED at ACCEPTED at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301) at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43) at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:492) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:70) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions$TestApplicationEventDispatcher.handle(TestRMAppTransitions.java:103) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions$TestApplicationEventDispatcher.handle(TestRMAppTransitions.java:89) at org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.InlineDispatcher.dispatch(InlineDispatcher.java:43) at org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.InlineDispatcher$TestEventHandler.handle(InlineDispatcher.java:33) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$BaseFinalTransition.transition(RMAppAttemptImpl.java:701) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$BaseFinalTransition.transition(RMAppAttemptImpl.java:651) at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357) at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298) at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43) at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:485) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.handle(RMAppAttemptImpl.java:80) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions$TestApplicationAttemptEventDispatcher.handle(TestRMAppTransitions.java:79) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions$TestApplicationAttemptEventDispatcher.handle(TestRMAppTransitions.java:65) at org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.InlineDispatcher.dispatch(InlineDispatcher.java:43) at org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.InlineDispatcher$TestEventHandler.handle(InlineDispatcher.java:33) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl$KillAppAndAttemptTransition.transition(RMAppImpl.java:563) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl$KillAppAndAttemptTransition.transition(RMAppImpl.java:559) at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357) at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298) at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43) at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:492) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:70) at org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions.testAppSubmittedKill(TestRMAppTransitions.java:369) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:228) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74) It would be nice if the state machine transition tests would fail on any invalid state transition encountered during the tests so these can be caught more proactively in the future.
        Mayank Bansal made changes -
        Field Original Value New Value
        Assignee Mayank Bansal [ mayank_bansal ]
        Hide
        Mayank Bansal added a comment -

        I think the issue is SUBMITTED And ACCEPTED APP transitions does not support for ATTEMPT_KILLED which is actuall being sent when KILL event is dispatched. Adding those two events.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - I think the issue is SUBMITTED And ACCEPTED APP transitions does not support for ATTEMPT_KILLED which is actuall being sent when KILL event is dispatched. Adding those two events. Thanks, Mayank
        Hide
        Mayank Bansal added a comment -

        Fixing the issue and test cases.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Fixing the issue and test cases. Thanks, Mayank
        Mayank Bansal made changes -
        Attachment MAPREDUCE-4455-trunk-v1.patch [ 12538971 ]
        Mayank Bansal made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s trunk [ 12320360 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12538971/MAPREDUCE-4455-trunk-v1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2701//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2701//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2701//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538971/MAPREDUCE-4455-trunk-v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2701//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2701//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2701//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        These find bug warnings are already addressed in
        MAPREDUCE-4452

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - These find bug warnings are already addressed in MAPREDUCE-4452 Thanks, Mayank
        Hide
        Siddharth Seth added a comment -

        Mayank, these transitions don't seem to be valid. I got this wrong while talking to you about this earlier... sorry about that.

        The unit test is broken in this case. The InlineDispatcher, which is used for this test and others, behaves differently from the regular dispatcher. It handles all events inline, as against handling one event to completion before processing the next one.
        For this test - the App should have transitioned to the KILLED state, before the ATTEMPT_KILLED event came in.
        The DrainDispatcher seems like a much better option for unit tests.

        Show
        Siddharth Seth added a comment - Mayank, these transitions don't seem to be valid. I got this wrong while talking to you about this earlier... sorry about that. The unit test is broken in this case. The InlineDispatcher, which is used for this test and others, behaves differently from the regular dispatcher. It handles all events inline, as against handling one event to completion before processing the next one. For this test - the App should have transitioned to the KILLED state, before the ATTEMPT_KILLED event came in. The DrainDispatcher seems like a much better option for unit tests.
        Siddharth Seth made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Target Version/s 2.2.0-alpha [ 12322471 ]
        Fix Version/s trunk [ 12320360 ]
        Hide
        Mayank Bansal added a comment -

        Thanks sid for your review comments. I incorporated those in latest patch.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Thanks sid for your review comments. I incorporated those in latest patch. Thanks, Mayank
        Mayank Bansal made changes -
        Attachment MAPREDUCE-4455-trunk-v2.patch [ 12540973 ]
        Mayank Bansal made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12540973/MAPREDUCE-4455-trunk-v2.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2729//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12540973/MAPREDUCE-4455-trunk-v2.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2729//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        ATtaching another patch with new file structure. I think we need to move this JIRA to YARN as well.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - ATtaching another patch with new file structure. I think we need to move this JIRA to YARN as well. Thanks, Mayank
        Mayank Bansal made changes -
        Attachment MAPREDUCE-4455-YARN-trunk-v2.patch [ 12541112 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12541112/MAPREDUCE-4455-YARN-trunk-v2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2738//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2738//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12541112/MAPREDUCE-4455-YARN-trunk-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2738//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2738//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        Thanks sid for your comments yesterday.

        Updated the patch with drain dispatcher and removed the sleep

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Thanks sid for your comments yesterday. Updated the patch with drain dispatcher and removed the sleep Thanks, Mayank
        Mayank Bansal made changes -
        Attachment MAPREDUCE-4455-YARN-trunk-v3.patch [ 12541263 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12541263/MAPREDUCE-4455-YARN-trunk-v3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2746//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2746//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12541263/MAPREDUCE-4455-YARN-trunk-v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2746//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2746//console This message is automatically generated.
        Siddharth Seth made changes -
        Project Hadoop Map/Reduce [ 12310941 ] Hadoop YARN [ 12313722 ]
        Key MAPREDUCE-4455 YARN-37
        Workflow no-reopen-closed, patch-avail [ 12717143 ] jira [ 12722114 ]
        Status Patch Available [ 10002 ] Open [ 1 ]
        Target Version/s 2.2.0-alpha [ 12322471 ]
        Component/s resourcemanager [ 12319322 ]
        Component/s mrv2 [ 12314301 ]
        Component/s resourcemanager [ 12315340 ]
        Hide
        Siddharth Seth added a comment -

        Mayank, thanks for the updated patch. Almost there, could do with one more minor change. All the tests end up generating an NPE at the end - since certain EventTypes are not registered to be handled (The InlineDispatcher used to ignore all of these). This does not really affect the tests - but it would be good to add NO-OP handlers for the additional EventTypes that are generated.

        Show
        Siddharth Seth added a comment - Mayank, thanks for the updated patch. Almost there, could do with one more minor change. All the tests end up generating an NPE at the end - since certain EventTypes are not registered to be handled (The InlineDispatcher used to ignore all of these). This does not really affect the tests - but it would be good to add NO-OP handlers for the additional EventTypes that are generated.
        Hide
        Mayank Bansal added a comment -

        Thanks sid for your comments.

        I added the required event handlers, I overlooked the Log messages.

        Thanks for pointing it out.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Thanks sid for your comments. I added the required event handlers, I overlooked the Log messages. Thanks for pointing it out. Thanks, Mayank
        Mayank Bansal made changes -
        Attachment YARN-37-trunk-v4.patch [ 12541839 ]
        Gavin made changes -
        Workflow jira [ 12722114 ] no-reopen-closed, patch-avail [ 12722161 ]
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Submitting to Jenkins to see if the commit-build I created is working properly or not.

        Show
        Vinod Kumar Vavilapalli added a comment - Submitting to Jenkins to see if the commit-build I created is working properly or not.
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Siddharth Seth added a comment -

        +1. Patch looks good. Waiting to see if Jenkins can be run for this, otherwise will commit it.

        Thanks Mayank

        Show
        Siddharth Seth added a comment - +1. Patch looks good. Waiting to see if Jenkins can be run for this, otherwise will commit it. Thanks Mayank
        Siddharth Seth made changes -
        Summary RMAppImpl state machine does not handle event ATTEMPT_KILLED at ACCEPTED TestRMAppTransitions.testAppSubmittedKilled passes for the wrong reason
        Affects Version/s 0.23.3 [ 12322841 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12541839/YARN-37-trunk-v4.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12541839/YARN-37-trunk-v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5//console This message is automatically generated.
        Hide
        Siddharth Seth added a comment -

        Committed to trunk, branch-2 and branch-2.1.0. Thanks Mayank

        Show
        Siddharth Seth added a comment - Committed to trunk, branch-2 and branch-2.1.0. Thanks Mayank
        Siddharth Seth made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 2.1.0-alpha [ 12322547 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2672 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2672/)
        YARN-37. Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal) (Revision 1377803)

        Result = FAILURE
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377803
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2672 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2672/ ) YARN-37 . Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal) (Revision 1377803) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377803 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2644 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2644/)
        YARN-37. Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal) (Revision 1377803)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377803
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2644 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2644/ ) YARN-37 . Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal) (Revision 1377803) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377803 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2708 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2708/)
        YARN-37. Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal) (Revision 1377803)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377803
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2708 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2708/ ) YARN-37 . Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal) (Revision 1377803) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377803 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1148 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1148/)
        YARN-37. Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal) (Revision 1377803)

        Result = FAILURE
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377803
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1148 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1148/ ) YARN-37 . Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal) (Revision 1377803) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377803 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1179 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1179/)
        YARN-37. Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal) (Revision 1377803)

        Result = FAILURE
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377803
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1179 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1179/ ) YARN-37 . Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal) (Revision 1377803) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377803 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Arun C Murthy made changes -
        Fix Version/s 2.0.2-alpha [ 12322548 ]
        Fix Version/s 2.1.0-alpha [ 12322547 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        17d 21h 8m 2 Siddharth Seth 21/Aug/12 22:58
        Open Open Patch Available Patch Available
        18d 2h 7m 3 Vinod Kumar Vavilapalli 23/Aug/12 20:14
        Patch Available Patch Available Resolved Resolved
        4d 25m 1 Siddharth Seth 27/Aug/12 20:40
        Resolved Resolved Closed Closed
        44d 22h 7m 1 Arun C Murthy 11/Oct/12 18:48

          People

          • Assignee:
            Mayank Bansal
            Reporter:
            Jason Lowe
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development