Whirr
  1. Whirr
  2. WHIRR-335

Update Hama service to Hama 0.4-incubating

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.2
    • Component/s: service/hama
    • Labels:
      None

      Description

      I'm planning to update Hama service to Hama 0.4-incubating.

      It'll contains many new features and major changes.

      1. whirr0.7.patch
        10 kB
        Edward J. Yoon
      2. final.patch
        13 kB
        Edward J. Yoon
      3. WHIRR-335.patch
        9 kB
        Edward J. Yoon

        Activity

        Andrei Savu made changes -
        Fix Version/s 0.8.0 [ 12318880 ]
        Hide
        Frank Scholten added a comment -

        Committed to trunk

        Show
        Frank Scholten added a comment - Committed to trunk
        Hide
        Andrei Savu added a comment -

        Frank can you also commit the other patch to trunk?

        Show
        Andrei Savu added a comment - Frank can you also commit the other patch to trunk?
        Hide
        Edward J. Yoon added a comment -

        Thanks Frank.

        Show
        Edward J. Yoon added a comment - Thanks Frank.
        Frank Scholten made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Frank Scholten added a comment -

        It's working. Committed to the 0.7 branch

        Show
        Frank Scholten added a comment - It's working. Committed to the 0.7 branch
        Hide
        Edward J. Yoon added a comment -

        Didn't know it required parent POM file. Please re-try build. I've deployed it to maven repository now.

        Show
        Edward J. Yoon added a comment - Didn't know it required parent POM file. Please re-try build. I've deployed it to maven repository now.
        Hide
        Frank Scholten added a comment -

        I applied the patch on 0.7 and get a "Could not find artifact org.apache.hama:hama-parent:pom:0.4.0-incubating" It's referenced by hama-core and I couldn't find hama-parent in mvn central.

        Show
        Frank Scholten added a comment - I applied the patch on 0.7 and get a "Could not find artifact org.apache.hama:hama-parent:pom:0.4.0-incubating" It's referenced by hama-core and I couldn't find hama-parent in mvn central.
        Edward J. Yoon made changes -
        Attachment whirr0.7.patch [ 12518124 ]
        Hide
        Edward J. Yoon added a comment -

        Here's patch for 0.7 branch.

        Builds and Integration tests are all passed.

        Show
        Edward J. Yoon added a comment - Here's patch for 0.7 branch. Builds and Integration tests are all passed.
        Hide
        Edward J. Yoon added a comment -

        Sure, I'll attach new patches.

        Show
        Edward J. Yoon added a comment - Sure, I'll attach new patches.
        Andrei Savu made changes -
        Fix Version/s 0.7.2 [ 12320175 ]
        Hide
        Andrei Savu added a comment - - edited

        Edward do you have some time to update this patch for branch 0.7?

        Show
        Andrei Savu added a comment - - edited Edward do you have some time to update this patch for branch 0.7?
        Hide
        Edward J. Yoon added a comment -

        Current 'HamaServiceTest' code is little messy but it'll be fixed with Hama 0.4.1-incubating. Also I'm plan on suppoting Hadoop 0.20.205.

        Show
        Edward J. Yoon added a comment - Current 'HamaServiceTest' code is little messy but it'll be fixed with Hama 0.4.1-incubating. Also I'm plan on suppoting Hadoop 0.20.205.
        Hide
        Andrei Savu added a comment -

        Great work! I will review & commit in a few minutes.

        Show
        Andrei Savu added a comment - Great work! I will review & commit in a few minutes.
        Edward J. Yoon made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Edward J. Yoon made changes -
        Attachment final.patch [ 12517394 ]
        Hide
        Edward J. Yoon added a comment -
        2012-03-07 21:39:43,775 INFO  [org.apache.whirr.service.hama.integration.HamaServiceController] (main) 1 groomservers reported in. Continuing.
        2012-03-07 21:39:43,813 INFO  [org.apache.whirr.service.hama.integration.HamaServiceTest] (Thread-37) Job conf: org.apache.hadoop.net.StandardSocketFactory, job__0000
        2012-03-07 21:39:43,816 INFO  [org.apache.whirr.service.hama.integration.HamaServiceTest] (Thread-37) Client configuration start ..
        2012-03-07 21:39:44,384 INFO  [org.apache.whirr.service.hama.integration.HamaServiceTest] (Thread-37) Client conf: org.apache.hadoop.net.SocksSocketFactory
        2012-03-07 21:40:05,477 INFO  [org.apache.whirr.service.hama.integration.HamaServiceTest] (Thread-37) finished
        2012-03-07 21:40:05,479 INFO  [org.apache.whirr.service.hama.integration.HamaServiceController] (main) Shutting down cluster...
        2012-03-07 21:40:05,495 INFO  [org.apache.whirr.actions.ScriptBasedClusterAction] (main) Finished running destroy phase scripts on all cluster instances
        2012-03-07 21:40:05,495 INFO  [org.apache.whirr.actions.DestroyClusterAction] (main) Destroying hama-itest-aws-ec2-edward cluster
        2012-03-07 21:40:48,623 INFO  [org.apache.whirr.actions.DestroyClusterAction] (main) Cluster hama-itest-aws-ec2-edward destroyed
        Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 429.088 sec
        

        Builds, Integration tests on EC2 passed.

        Show
        Edward J. Yoon added a comment - 2012-03-07 21:39:43,775 INFO [org.apache.whirr.service.hama.integration.HamaServiceController] (main) 1 groomservers reported in. Continuing. 2012-03-07 21:39:43,813 INFO [org.apache.whirr.service.hama.integration.HamaServiceTest] ( Thread -37) Job conf: org.apache.hadoop.net.StandardSocketFactory, job__0000 2012-03-07 21:39:43,816 INFO [org.apache.whirr.service.hama.integration.HamaServiceTest] ( Thread -37) Client configuration start .. 2012-03-07 21:39:44,384 INFO [org.apache.whirr.service.hama.integration.HamaServiceTest] ( Thread -37) Client conf: org.apache.hadoop.net.SocksSocketFactory 2012-03-07 21:40:05,477 INFO [org.apache.whirr.service.hama.integration.HamaServiceTest] ( Thread -37) finished 2012-03-07 21:40:05,479 INFO [org.apache.whirr.service.hama.integration.HamaServiceController] (main) Shutting down cluster... 2012-03-07 21:40:05,495 INFO [org.apache.whirr.actions.ScriptBasedClusterAction] (main) Finished running destroy phase scripts on all cluster instances 2012-03-07 21:40:05,495 INFO [org.apache.whirr.actions.DestroyClusterAction] (main) Destroying hama-itest-aws-ec2-edward cluster 2012-03-07 21:40:48,623 INFO [org.apache.whirr.actions.DestroyClusterAction] (main) Cluster hama-itest-aws-ec2-edward destroyed Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 429.088 sec Builds, Integration tests on EC2 passed.
        Hide
        Andrei Savu added a comment -

        Note: if you are having issue with Hadoop try to avoid archive.apache.org and use a different mirror.

        Show
        Andrei Savu added a comment - Note: if you are having issue with Hadoop try to avoid archive.apache.org and use a different mirror.
        Edward J. Yoon made changes -
        Attachment WHIRR-335.patch [ 12517234 ]
        Hide
        Edward J. Yoon added a comment -

        The attached patch updates tarball-urls and adds whirr.hadoop.version to default-properties. (we don't support hadoop-0.20.205 yet).

        However, integration test still fails. It looks like a bug of HamaServiceTest. I'll fix soon.

        edward@ip-10-244-141-222:/usr/local/hama-0.4.0-incubating$ bin/hama jar hama-examples-0.4.0-incubating.jar pi
        12/03/06 13:05:24 INFO bsp.BSPJobClient: Running job: job_201203061255_0002
        12/03/06 13:05:27 INFO bsp.BSPJobClient: Current supersteps number: 0
        12/03/06 13:05:33 INFO bsp.BSPJobClient: Current supersteps number: 1
        12/03/06 13:05:33 INFO bsp.BSPJobClient: The total number of supersteps: 1
        Estimated value of PI is	3.1697333333333333
        Job Finished in 9.644 seconds
        edward@ip-10-244-141-222:/usr/local/hama-0.4.0-incubating$ bin/hama job -list all
        2 jobs submitted
        States are:
        	Running : 1	Succeded : 2	Failed : 3	Prep : 4
        JobId	State	StartTime	UserName
        job_201203061255_0001	1	1331038583285	edward
        job_201203061255_0002	2	1331039124255	edward
        edward@ip-10-244-141-222:/usr/local/hama-0.4.0-incubating$ bin/hama job -kill job_201203061255_0001
        Killed job job_201203061255_0001
        edward@ip-10-244-141-222:/usr/local/hama-0.4.0-incubating$ 
        
        Show
        Edward J. Yoon added a comment - The attached patch updates tarball-urls and adds whirr.hadoop.version to default-properties. (we don't support hadoop-0.20.205 yet). However, integration test still fails. It looks like a bug of HamaServiceTest. I'll fix soon. edward@ip-10-244-141-222:/usr/local/hama-0.4.0-incubating$ bin/hama jar hama-examples-0.4.0-incubating.jar pi 12/03/06 13:05:24 INFO bsp.BSPJobClient: Running job: job_201203061255_0002 12/03/06 13:05:27 INFO bsp.BSPJobClient: Current supersteps number: 0 12/03/06 13:05:33 INFO bsp.BSPJobClient: Current supersteps number: 1 12/03/06 13:05:33 INFO bsp.BSPJobClient: The total number of supersteps: 1 Estimated value of PI is 3.1697333333333333 Job Finished in 9.644 seconds edward@ip-10-244-141-222:/usr/local/hama-0.4.0-incubating$ bin/hama job -list all 2 jobs submitted States are: Running : 1 Succeded : 2 Failed : 3 Prep : 4 JobId State StartTime UserName job_201203061255_0001 1 1331038583285 edward job_201203061255_0002 2 1331039124255 edward edward@ip-10-244-141-222:/usr/local/hama-0.4.0-incubating$ bin/hama job -kill job_201203061255_0001 Killed job job_201203061255_0001 edward@ip-10-244-141-222:/usr/local/hama-0.4.0-incubating$
        Edward J. Yoon made changes -
        Fix Version/s 0.8.0 [ 12318880 ]
        Fix Version/s 0.9.0 [ 12319840 ]
        Hide
        Edward J. Yoon added a comment -

        Change Fix Version/s to 0.8.

        Hama 0.4 released out. And, currently Hama service integration test fails.

        Show
        Edward J. Yoon added a comment - Change Fix Version/s to 0.8. Hama 0.4 released out. And, currently Hama service integration test fails.
        Andrei Savu made changes -
        Fix Version/s 0.9.0 [ 12319840 ]
        Fix Version/s 0.8.0 [ 12318880 ]
        Hide
        Edward J. Yoon added a comment -

        yes.

        Show
        Edward J. Yoon added a comment - yes.
        Hide
        Andrei Savu added a comment -

        Is the Hama release going to happen in the next 2-3 weeks?

        Show
        Andrei Savu added a comment - Is the Hama release going to happen in the next 2-3 weeks?
        Edward J. Yoon made changes -
        Field Original Value New Value
        Fix Version/s 0.8.0 [ 12318880 ]
        Component/s service/hama [ 12315327 ]
        Edward J. Yoon created issue -

          People

          • Assignee:
            Edward J. Yoon
            Reporter:
            Edward J. Yoon
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development