Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4703

Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 2.0.3-alpha
    • Fix Version/s: 1.2.0, 2.0.3-alpha
    • Component/s: mrv1, mrv2, test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The objective here is to enable starting back the cluster, after being stopped, using the same configurations/port numbers used before stopping.

      1. MAPREDUCE-4703.patch
        8 kB
        Ahmed Radwan
      2. MAPREDUCE-4703_rev3.patch
        8 kB
        Ahmed Radwan
      3. MAPREDUCE-4703_rev2.patch
        8 kB
        Ahmed Radwan
      4. MAPREDUCE-4703_branch-1.patch
        4 kB
        Ahmed Radwan
      5. MAPREDUCE-4703_branch-1_rev4.patch
        4 kB
        Ahmed Radwan
      6. MAPREDUCE-4703_branch-1_rev3.patch
        4 kB
        Ahmed Radwan
      7. MAPREDUCE-4703_branch-1_rev2.patch
        4 kB
        Ahmed Radwan

        Activity

        Ahmed Radwan created issue -
        Hide
        Ahmed Radwan added a comment -

        Here is a patch for trunk.

        Show
        Ahmed Radwan added a comment - Here is a patch for trunk.
        Ahmed Radwan made changes -
        Field Original Value New Value
        Attachment MAPREDUCE-4703.patch [ 12547662 ]
        Ahmed Radwan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12547662/MAPREDUCE-4703.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2906//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2906//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12547662/MAPREDUCE-4703.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2906//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2906//console This message is automatically generated.
        Hide
        Alejandro Abdelnur added a comment -

        It looks good, only NIT is that the MiniMRClientCluster class & methods should be properly javadoc, and the restart() method should clearly state that it must be stopped before calling restart() else people may assume the method will do a stop/start.

        Show
        Alejandro Abdelnur added a comment - It looks good, only NIT is that the MiniMRClientCluster class & methods should be properly javadoc, and the restart() method should clearly state that it must be stopped before calling restart() else people may assume the method will do a stop/start.
        Robert Kanter made changes -
        Link This issue blocks OOZIE-1011 [ OOZIE-1011 ]
        Hide
        Robert Kanter added a comment -

        Is this different from MiniMRCluster? Oozie needs this functionality to fix some tests for OOZIE-994 but its using MiniMRCluster, not MiniMRClientCluster.

        Show
        Robert Kanter added a comment - Is this different from MiniMRCluster ? Oozie needs this functionality to fix some tests for OOZIE-994 but its using MiniMRCluster , not MiniMRClientCluster .
        Hide
        Ahmed Radwan added a comment -

        In MR2, MiniMRCluster is just a wrapper around MiniMRClientCluster. I will also expose the restart() method through MiniMRCluster so older tests doesn't need to switch. For new tests, they can directly use the MiniMRClientCluster. I am also writing the corresponding patch for branch-1

        Show
        Ahmed Radwan added a comment - In MR2, MiniMRCluster is just a wrapper around MiniMRClientCluster. I will also expose the restart() method through MiniMRCluster so older tests doesn't need to switch. For new tests, they can directly use the MiniMRClientCluster. I am also writing the corresponding patch for branch-1
        Hide
        Ahmed Radwan added a comment -

        Here is the branch-1 batch.

        Show
        Ahmed Radwan added a comment - Here is the branch-1 batch.
        Ahmed Radwan made changes -
        Attachment MAPREDUCE-4703_branch-1.patch [ 12549477 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12549477/MAPREDUCE-4703_branch-1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2934//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12549477/MAPREDUCE-4703_branch-1.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2934//console This message is automatically generated.
        Hide
        Ahmed Radwan added a comment -

        I have also updated the trunk patch (similar to branch-1) to have the new restart() method stopping and then starting the cluster using the same configurations.

        Show
        Ahmed Radwan added a comment - I have also updated the trunk patch (similar to branch-1) to have the new restart() method stopping and then starting the cluster using the same configurations.
        Ahmed Radwan made changes -
        Attachment MAPREDUCE-4703_rev2.patch [ 12549482 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12549482/MAPREDUCE-4703_rev2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2935//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2935//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12549482/MAPREDUCE-4703_rev2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2935//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2935//console This message is automatically generated.
        Robert Kanter made changes -
        Link This issue blocks OOZIE-1011 [ OOZIE-1011 ]
        Hide
        Alejandro Abdelnur added a comment -

        It looks good, a couple of nits:

        • remove the javadocs that only says "@throw Exception" in MiniMRClientCluster, and keep the explanation of the restart.
        • the formatting indentation of the restart() method is funny, you have 6 more chars to go before reaching 80 and it would be much easier to read, (also, IMO, for 1 or 2 chars, no need to go down if gets ugly to the eye).
        Show
        Alejandro Abdelnur added a comment - It looks good, a couple of nits: remove the javadocs that only says "@throw Exception" in MiniMRClientCluster, and keep the explanation of the restart. the formatting indentation of the restart() method is funny, you have 6 more chars to go before reaching 80 and it would be much easier to read, (also, IMO, for 1 or 2 chars, no need to go down if gets ugly to the eye).
        Robert Kanter made changes -
        Attachment MAPREDUCE-4703_branch-1.patch [ 12554604 ]
        Attachment MAPREDUCE-4703.patch [ 12554605 ]
        Robert Kanter made changes -
        Assignee Ahmed Radwan [ ahmed.radwan ] Robert Kanter [ rkanter ]
        Hide
        Ahmed Radwan added a comment -

        Thanks Tucu for reviewing the patch! I'll take care of these nits and upload new patches.

        Show
        Ahmed Radwan added a comment - Thanks Tucu for reviewing the patch! I'll take care of these nits and upload new patches.
        Robert Kanter made changes -
        Assignee Robert Kanter [ rkanter ] Ahmed Radwan [ ahmed.radwan ]
        Robert Kanter made changes -
        Attachment MAPREDUCE-4703_branch-1.patch [ 12554604 ]
        Robert Kanter made changes -
        Attachment MAPREDUCE-4703.patch [ 12554605 ]
        Alejandro Abdelnur made changes -
        Comment [ This was Ahmed's patch, but I don't think he's going to be working on it anymore, so I uploaded updated patches that address the javadoc and formatting. ]
        Hide
        Robert Kanter added a comment -

        Sorry Ahmed, I had posted an updated patch without knowing you'd work on it. I've reassigned it back to you.

        Show
        Robert Kanter added a comment - Sorry Ahmed, I had posted an updated patch without knowing you'd work on it. I've reassigned it back to you.
        Hide
        Ahmed Radwan added a comment -

        Thanks Robert! Sorry I didn't notice that. I'll be updating the patches later today and see if Tucu will have more comments.

        Show
        Ahmed Radwan added a comment - Thanks Robert! Sorry I didn't notice that. I'll be updating the patches later today and see if Tucu will have more comments.
        Hide
        Ahmed Radwan added a comment -

        Here are the updated patches.

        Show
        Ahmed Radwan added a comment - Here are the updated patches.
        Ahmed Radwan made changes -
        Attachment MAPREDUCE-4703_branch-1_rev2.patch [ 12554641 ]
        Ahmed Radwan made changes -
        Attachment MAPREDUCE-4703_rev3.patch [ 12554642 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12554642/MAPREDUCE-4703_rev3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3054//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3054//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554642/MAPREDUCE-4703_rev3.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3054//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3054//console This message is automatically generated.
        Alejandro Abdelnur made changes -
        Issue Type Bug [ 1 ] Improvement [ 4 ]
        Hide
        Alejandro Abdelnur added a comment -

        Thanks Ahmed. I've committed to trunk and branch-2 after running the tests.

        However, when trying to run the test with the branch-1 patch the test is failing with the following output.

        Would you please take a look at it? I'll hold off committing to branch-1 until this is addressed. Leaving the JIRA open as well.

        Testcase: testJob took 14.387 sec
        Testcase: testRestart took 7.333 sec
        	Caused an ERROR
        java.io.IOException: Call to localhost/127.0.0.1:59747 failed on local exception: java.io.EOFException
        java.lang.RuntimeException: java.io.IOException: Call to localhost/127.0.0.1:59747 failed on local exception: java.io.EOFException
        	at org.apache.hadoop.mapred.MiniMRCluster.waitUntilIdle(MiniMRCluster.java:325)
        	at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:527)
        	at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:465)
        	at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:457)
        	at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:449)
        	at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:439)
        	at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:429)
        	at org.apache.hadoop.mapred.MiniMRClusterAdapter.restart(MiniMRClusterAdapter.java:80)
        	at org.apache.hadoop.mapred.TestMiniMRClientCluster.testRestart(TestMiniMRClientCluster.java:109)
        Caused by: java.io.IOException: Call to localhost/127.0.0.1:59747 failed on local exception: java.io.EOFException
        	at org.apache.hadoop.ipc.Client.wrapException(Client.java:1144)
        	at org.apache.hadoop.ipc.Client.call(Client.java:1112)
        	at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
        	at org.apache.hadoop.mapred.$Proxy10.getProtocolVersion(Unknown Source)
        	at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:411)
        	at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:505)
        	at org.apache.hadoop.mapred.JobClient.init(JobClient.java:496)
        	at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:479)
        	at org.apache.hadoop.mapred.MiniMRCluster.waitUntilIdle(MiniMRCluster.java:311)
        Caused by: java.io.EOFException
        	at java.io.DataInputStream.readInt(DataInputStream.java:375)
        	at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:841)
        	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:786)
        
        Show
        Alejandro Abdelnur added a comment - Thanks Ahmed. I've committed to trunk and branch-2 after running the tests. However, when trying to run the test with the branch-1 patch the test is failing with the following output. Would you please take a look at it? I'll hold off committing to branch-1 until this is addressed. Leaving the JIRA open as well. Testcase: testJob took 14.387 sec Testcase: testRestart took 7.333 sec Caused an ERROR java.io.IOException: Call to localhost/127.0.0.1:59747 failed on local exception: java.io.EOFException java.lang.RuntimeException: java.io.IOException: Call to localhost/127.0.0.1:59747 failed on local exception: java.io.EOFException at org.apache.hadoop.mapred.MiniMRCluster.waitUntilIdle(MiniMRCluster.java:325) at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:527) at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:465) at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:457) at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:449) at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:439) at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:429) at org.apache.hadoop.mapred.MiniMRClusterAdapter.restart(MiniMRClusterAdapter.java:80) at org.apache.hadoop.mapred.TestMiniMRClientCluster.testRestart(TestMiniMRClientCluster.java:109) Caused by: java.io.IOException: Call to localhost/127.0.0.1:59747 failed on local exception: java.io.EOFException at org.apache.hadoop.ipc.Client.wrapException(Client.java:1144) at org.apache.hadoop.ipc.Client.call(Client.java:1112) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229) at org.apache.hadoop.mapred.$Proxy10.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:411) at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:505) at org.apache.hadoop.mapred.JobClient.init(JobClient.java:496) at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:479) at org.apache.hadoop.mapred.MiniMRCluster.waitUntilIdle(MiniMRCluster.java:311) Caused by: java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:375) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:841) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:786)
        Alejandro Abdelnur made changes -
        Fix Version/s 2.0.3-alpha [ 12323275 ]
        Affects Version/s 1.2.0 [ 12321661 ]
        Affects Version/s 2.0.3-alpha [ 12323275 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3102 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3102/)
        MAPREDUCE-4703. Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. (ahmed.radwan via tucu) (Revision 1419618)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1419618
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientCluster.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientClusterFactory.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRYarnClusterAdapter.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3102 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3102/ ) MAPREDUCE-4703 . Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. (ahmed.radwan via tucu) (Revision 1419618) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1419618 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientCluster.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientClusterFactory.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRYarnClusterAdapter.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
        Hide
        Ahmed Radwan added a comment -

        Thanks Tucu! I'll take a look at this failure and get back to you.

        Show
        Ahmed Radwan added a comment - Thanks Tucu! I'll take a look at this failure and get back to you.
        Hide
        Ahmed Radwan added a comment -

        The branch-1 patch runs fine for me. I am only seeing this same failure when reducing the 5 sec sleep period that takes place after shutting down the cluster and before starting. This sleep period is machine dependent and this is why you are seeing it and I am not. I have modified the patch to avoid having a fixed sleep period, instead it now retries to start the cluster every 1 sec until it starts. Can you please test this patch on your machine and see if it solves the problem?

        Show
        Ahmed Radwan added a comment - The branch-1 patch runs fine for me. I am only seeing this same failure when reducing the 5 sec sleep period that takes place after shutting down the cluster and before starting. This sleep period is machine dependent and this is why you are seeing it and I am not. I have modified the patch to avoid having a fixed sleep period, instead it now retries to start the cluster every 1 sec until it starts. Can you please test this patch on your machine and see if it solves the problem?
        Ahmed Radwan made changes -
        Attachment MAPREDUCE-4703_branch-1_rev3.patch [ 12560370 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12560370/MAPREDUCE-4703_branch-1_rev3.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3116//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12560370/MAPREDUCE-4703_branch-1_rev3.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3116//console This message is automatically generated.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #62 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/62/)
        MAPREDUCE-4703. Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. (ahmed.radwan via tucu) (Revision 1419618)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1419618
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientCluster.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientClusterFactory.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRYarnClusterAdapter.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #62 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/62/ ) MAPREDUCE-4703 . Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. (ahmed.radwan via tucu) (Revision 1419618) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1419618 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientCluster.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientClusterFactory.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRYarnClusterAdapter.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1251 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1251/)
        MAPREDUCE-4703. Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. (ahmed.radwan via tucu) (Revision 1419618)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1419618
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientCluster.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientClusterFactory.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRYarnClusterAdapter.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1251 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1251/ ) MAPREDUCE-4703 . Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. (ahmed.radwan via tucu) (Revision 1419618) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1419618 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientCluster.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientClusterFactory.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRYarnClusterAdapter.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1282 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1282/)
        MAPREDUCE-4703. Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. (ahmed.radwan via tucu) (Revision 1419618)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1419618
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientCluster.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientClusterFactory.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRYarnClusterAdapter.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1282 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1282/ ) MAPREDUCE-4703 . Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. (ahmed.radwan via tucu) (Revision 1419618) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1419618 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientCluster.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRClientClusterFactory.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MiniMRYarnClusterAdapter.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
        Hide
        Alejandro Abdelnur added a comment -

        Ahmed, thanks for taking care of this. One last ask, instead doing a while(true) attempting restart, could we use a counter with a reasonable high number, ie 30, to bail out? Else we'll be stuck until the build time out for tests kicks in.

        Show
        Alejandro Abdelnur added a comment - Ahmed, thanks for taking care of this. One last ask, instead doing a while(true) attempting restart, could we use a counter with a reasonable high number, ie 30, to bail out? Else we'll be stuck until the build time out for tests kicks in.
        Hide
        Ahmed Radwan added a comment -

        Thanks Tucu, I have modified the patch per your suggestion to retry for a max. of 30 sec. instead of relying on the test timeout. Please take a look on the updated patch.

        Show
        Ahmed Radwan added a comment - Thanks Tucu, I have modified the patch per your suggestion to retry for a max. of 30 sec. instead of relying on the test timeout. Please take a look on the updated patch.
        Ahmed Radwan made changes -
        Attachment MAPREDUCE-4703_branch-1_rev4.patch [ 12561166 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12561166/MAPREDUCE-4703_branch-1_rev4.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3126//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12561166/MAPREDUCE-4703_branch-1_rev4.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3126//console This message is automatically generated.
        Hide
        Robert Kanter added a comment -

        The rev4 branch-1 patch works for me

        Show
        Robert Kanter added a comment - The rev4 branch-1 patch works for me
        Hide
        Alejandro Abdelnur added a comment -

        +1

        Show
        Alejandro Abdelnur added a comment - +1
        Hide
        Alejandro Abdelnur added a comment -

        Thanks Ahmed. Committed to branch-1 as well.

        Show
        Alejandro Abdelnur added a comment - Thanks Ahmed. Committed to branch-1 as well.
        Alejandro Abdelnur made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 1.2.0 [ 12321661 ]
        Resolution Fixed [ 1 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Ahmed Radwan
            Reporter:
            Ahmed Radwan
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development