• Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: 0.7.0
    • Component/s: new service
    • Labels:


      Here is an initial patch to support Mahout as a Whirr service.

      I created the role 'mahout-home' which can be used to install the binary Mahout distribution on a Hadoop namenode.
      By combining this role with configuration for a Hadoop cluster you can SSH into the namenode, su to root and start running Mahout jobs via the mahout script immediately.

      The 'mahout-home' role has two properties

      Mahout version whirr.mahout.version
      URL of the Mahout binary distribution tarball whirr.mahout.tarball.url

      Note that I used a snapshot version of Mahout for testing, revision 1169784, because there were some problems with the Mahout script in 0.5 that have been fixed on trunk, see MAHOUT-680. To test you can set the tarball property to this link

      I used configure actions and the onBeforeConfigure(). If there is a better way to express this with the Whirr API let me know.

      Currently I am investigating a 'mahout-jar' role, which installs the Mahout examples job jar under $HADOOP_HOME/lib on a tasktracer node. I already have some code for putting the jar in place but when running a job from my local machine I still get ClassNotFoundExceptions. I believe this is because Hadoop has already started before the jar is put in the lib dir, so the jar won't be picked up, but I have to investigate some more. From WHIRR-221 I understood that there is no support (yet?) for ordering of services but if you have an idea on how to fix this let me know.

      Comments and suggestions welcome!

      1. WHIRR-384.patch
        27 kB
        Andrei Savu
      2. WHIRR-384-mahout-client.patch
        12 kB
        Frank Scholten
      3. WHIRR-384-mahout-home.patch
        13 kB
        Frank Scholten


        Frank Scholten created issue -
        Frank Scholten made changes -
        Field Original Value New Value
        Attachment WHIRR-384-mahout-home.patch [ 12494124 ]
        Frank Scholten made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Frank Scholten made changes -
        Attachment WHIRR-384-mahout-client.patch [ 12494307 ]
        Andrei Savu made changes -
        Fix Version/s 0.8.0 [ 12318880 ]
        Fix Version/s 0.7.0 [ 12317571 ]
        Andrei Savu made changes -
        Attachment WHIRR-384.patch [ 12506732 ]
        Andrei Savu made changes -
        Assignee Frank Scholten [ frankscholten ]
        Andrei Savu made changes -
        Fix Version/s 0.7.0 [ 12317571 ]
        Fix Version/s 0.8.0 [ 12318880 ]
        Andrei Savu made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]


          • Assignee:
            Frank Scholten
            Frank Scholten
          • Votes:
            0 Vote for this issue
            1 Start watching this issue


            • Created: