Mahout
  1. Mahout
  2. MAHOUT-110

Ant script for building Taste web app

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.1
    • Fix Version/s: 0.1
    • Labels:
      None

      Description

      WIll attach patch after creating. This is a follow-up from a thread on mahout-dev.

      1. AntScript.patch
        11 kB
        Sean Owen
      2. MAHOUT-110.patch
        8 kB
        Grant Ingersoll
      3. MAHOUT-110.patch
        5 kB
        Grant Ingersoll
      4. MAHOUT-110-docs.patch
        4 kB
        Grant Ingersoll

        Issue Links

          Activity

          Hide
          Grant Ingersoll added a comment -

          It seems like the only thing this offers above the POM is the fact that it can build the grouplens example right?

          How about if we had the WAR built and then just copied it to create a group lens version (via Ant)?

          In other words, I can hook into the package goal for Maven a call to the Ant script to build a Group lens version of the WAR file.

          Would that work? If you can do the Ant part, I can hook it in.

          Show
          Grant Ingersoll added a comment - It seems like the only thing this offers above the POM is the fact that it can build the grouplens example right? How about if we had the WAR built and then just copied it to create a group lens version (via Ant)? In other words, I can hook into the package goal for Maven a call to the Ant script to build a Group lens version of the WAR file. Would that work? If you can do the Ant part, I can hook it in.
          Hide
          Grant Ingersoll added a comment -

          When I run "mvn package" in taste-web, here is what I get in the WAR:
          META-INF/
          META-INF/MANIFEST.MF
          WEB-INF/
          WEB-INF/classes/
          WEB-INF/classes/META-INF/
          WEB-INF/classes/org/
          WEB-INF/classes/org/apache/
          WEB-INF/classes/org/apache/mahout/
          WEB-INF/classes/org/apache/mahout/cf/
          WEB-INF/classes/org/apache/mahout/cf/taste/
          WEB-INF/classes/org/apache/mahout/cf/taste/web/
          WEB-INF/lib/
          WEB-INF/classes/META-INF/LICENSE.txt
          WEB-INF/classes/META-INF/NOTICE.txt
          WEB-INF/classes/META-INF/README.txt
          WEB-INF/classes/org/apache/mahout/cf/taste/web/RecommenderServlet.class
          WEB-INF/classes/org/apache/mahout/cf/taste/web/RecommenderSingleton.class
          WEB-INF/lib/activation-1.1.jar
          WEB-INF/lib/axis-1.4.jar
          WEB-INF/lib/axis-jaxrpc-1.4.jar
          WEB-INF/lib/axis-saaj-1.4.jar
          WEB-INF/lib/axis-wsdl4j-1.5.1.jar
          WEB-INF/lib/commons-cli-2.0-mahout.jar
          WEB-INF/lib/commons-codec-1.2.jar
          WEB-INF/lib/commons-dbcp-1.2.2.jar
          WEB-INF/lib/commons-discovery-0.2.jar
          WEB-INF/lib/commons-httpclient-3.1.jar
          WEB-INF/lib/commons-lang-2.4.jar
          WEB-INF/lib/commons-logging-1.1.1.jar
          WEB-INF/lib/commons-pool-1.4.jar
          WEB-INF/lib/gson-1.3.jar
          WEB-INF/lib/hadoop-core-0.19.1.jar
          WEB-INF/lib/jets3t-0.6.1.jar
          WEB-INF/lib/kfs-0.2.1.jar
          WEB-INF/lib/log4j-1.2.15.jar
          WEB-INF/lib/lucene-analyzers-2.3.2.jar
          WEB-INF/lib/lucene-core-2.3.2.jar
          WEB-INF/lib/mahout-core-0.1-SNAPSHOT.jar
          WEB-INF/lib/mail-1.4.jar
          WEB-INF/lib/servlet-api-2.4.jar
          WEB-INF/lib/slf4j-api-1.5.5.jar
          WEB-INF/lib/slf4j-jcl-1.5.5.jar
          WEB-INF/lib/uncommons-math-1.0.2.jar
          WEB-INF/lib/watchmaker-framework-0.5.0.jar
          WEB-INF/lib/watchmaker-swing-0.5.0.jar
          WEB-INF/lib/xmlenc-0.52.jar
          WEB-INF/lib/xpp3_min-1.1.3.4.O.jar
          WEB-INF/lib/xstream-1.2.1.jar
          WEB-INF/RecommenderService.jws
          WEB-INF/web.xml
          META-INF/maven/
          META-INF/maven/org.apache.mahout/
          META-INF/maven/org.apache.mahout/mahout-taste-webapp/
          META-INF/maven/org.apache.mahout/mahout-taste-webapp/pom.xml
          META-INF/maven/org.apache.mahout/mahout-taste-webapp/pom.properties

          Show
          Grant Ingersoll added a comment - When I run "mvn package" in taste-web, here is what I get in the WAR: META-INF/ META-INF/MANIFEST.MF WEB-INF/ WEB-INF/classes/ WEB-INF/classes/META-INF/ WEB-INF/classes/org/ WEB-INF/classes/org/apache/ WEB-INF/classes/org/apache/mahout/ WEB-INF/classes/org/apache/mahout/cf/ WEB-INF/classes/org/apache/mahout/cf/taste/ WEB-INF/classes/org/apache/mahout/cf/taste/web/ WEB-INF/lib/ WEB-INF/classes/META-INF/LICENSE.txt WEB-INF/classes/META-INF/NOTICE.txt WEB-INF/classes/META-INF/README.txt WEB-INF/classes/org/apache/mahout/cf/taste/web/RecommenderServlet.class WEB-INF/classes/org/apache/mahout/cf/taste/web/RecommenderSingleton.class WEB-INF/lib/activation-1.1.jar WEB-INF/lib/axis-1.4.jar WEB-INF/lib/axis-jaxrpc-1.4.jar WEB-INF/lib/axis-saaj-1.4.jar WEB-INF/lib/axis-wsdl4j-1.5.1.jar WEB-INF/lib/commons-cli-2.0-mahout.jar WEB-INF/lib/commons-codec-1.2.jar WEB-INF/lib/commons-dbcp-1.2.2.jar WEB-INF/lib/commons-discovery-0.2.jar WEB-INF/lib/commons-httpclient-3.1.jar WEB-INF/lib/commons-lang-2.4.jar WEB-INF/lib/commons-logging-1.1.1.jar WEB-INF/lib/commons-pool-1.4.jar WEB-INF/lib/gson-1.3.jar WEB-INF/lib/hadoop-core-0.19.1.jar WEB-INF/lib/jets3t-0.6.1.jar WEB-INF/lib/kfs-0.2.1.jar WEB-INF/lib/log4j-1.2.15.jar WEB-INF/lib/lucene-analyzers-2.3.2.jar WEB-INF/lib/lucene-core-2.3.2.jar WEB-INF/lib/mahout-core-0.1-SNAPSHOT.jar WEB-INF/lib/mail-1.4.jar WEB-INF/lib/servlet-api-2.4.jar WEB-INF/lib/slf4j-api-1.5.5.jar WEB-INF/lib/slf4j-jcl-1.5.5.jar WEB-INF/lib/uncommons-math-1.0.2.jar WEB-INF/lib/watchmaker-framework-0.5.0.jar WEB-INF/lib/watchmaker-swing-0.5.0.jar WEB-INF/lib/xmlenc-0.52.jar WEB-INF/lib/xpp3_min-1.1.3.4.O.jar WEB-INF/lib/xstream-1.2.1.jar WEB-INF/RecommenderService.jws WEB-INF/web.xml META-INF/maven/ META-INF/maven/org.apache.mahout/ META-INF/maven/org.apache.mahout/mahout-taste-webapp/ META-INF/maven/org.apache.mahout/mahout-taste-webapp/pom.xml META-INF/maven/org.apache.mahout/mahout-taste-webapp/pom.properties
          Hide
          Sean Owen added a comment -

          No, to be clear, this works for no situations at all – which includes the GroupLens example. The generated .war contains no Recommender that can be used with this package.

          If you've got a quick fix for the pom.xml that lets callers specify a .jar file, and include in the .war's WEB-INF/lib, that should fix all of this.

          I'd not spend too much time on this – we've lost a fair bit of time on this already. I need this mechanism to work, and have a working patch. I'd like it to work by the end of the week so we can move on at last with 0.1. I think that means committing this patch or a work-alike change to pom.xml.

          Show
          Sean Owen added a comment - No, to be clear, this works for no situations at all – which includes the GroupLens example. The generated .war contains no Recommender that can be used with this package. If you've got a quick fix for the pom.xml that lets callers specify a .jar file, and include in the .war's WEB-INF/lib, that should fix all of this. I'd not spend too much time on this – we've lost a fair bit of time on this already. I need this mechanism to work, and have a working patch. I'd like it to work by the end of the week so we can move on at last with 0.1. I think that means committing this patch or a work-alike change to pom.xml.
          Hide
          Sean Owen added a comment -

          I'd also note that this .war file contains a ton more stuff than it needs like Hadoop and GSon libs. It won't hurt, but is a bit messy. It's another of these things that is nagging me about Maven... it just isn't the right tool for this particular part of the build. Hence, the Ant script that brings this forward (back?) to a clean, working state. I am hearing support to proceed with this – the last question is how much of a pom.xml must be left to satisfy Maven's packaging requirements.

          Show
          Sean Owen added a comment - I'd also note that this .war file contains a ton more stuff than it needs like Hadoop and GSon libs. It won't hurt, but is a bit messy. It's another of these things that is nagging me about Maven... it just isn't the right tool for this particular part of the build. Hence, the Ant script that brings this forward (back?) to a clean, working state. I am hearing support to proceed with this – the last question is how much of a pom.xml must be left to satisfy Maven's packaging requirements.
          Hide
          Grant Ingersoll added a comment -

          I'd also note that this .war file contains a ton more stuff than it needs like Hadoop and GSon libs
          {quote]

          The real answer to this, of course, is to split out Taste from the core and have it be it's own module.

          If you've got a quick fix for the pom.xml that lets callers specify a .jar file, and include in the .war's WEB-INF/lib, that should fix all of this.

          OK. I'll take a look. Is there a default one that people can use? I suppose if I make it work for Grouplens, that would suffice.

          I agree with your timeframe, I'm anxious to get 0.1 out too, since I need to stand up in front of a bunch of people next week and explain Mahout. In fact, I'd love to be able to demo Taste! I'm working on this today.

          Show
          Grant Ingersoll added a comment - I'd also note that this .war file contains a ton more stuff than it needs like Hadoop and GSon libs {quote] The real answer to this, of course, is to split out Taste from the core and have it be it's own module. If you've got a quick fix for the pom.xml that lets callers specify a .jar file, and include in the .war's WEB-INF/lib, that should fix all of this. OK. I'll take a look. Is there a default one that people can use? I suppose if I make it work for Grouplens, that would suffice. I agree with your timeframe, I'm anxious to get 0.1 out too, since I need to stand up in front of a bunch of people next week and explain Mahout. In fact, I'd love to be able to demo Taste! I'm working on this today.
          Hide
          Sean Owen added a comment -

          No, to be clear, this works for no situations at all – which includes the GroupLens example. The generated .war contains no Recommender that can be used with this package.

          If you've got a quick fix for the pom.xml that lets callers specify a .jar file, and include in the .war's WEB-INF/lib, that should fix all of this.

          I'd not spend too much time on this – we've lost a fair bit of time on this already. I need this mechanism to work, and have a working patch. I'd like it to work by the end of the week so we can move on at last with 0.1. I think that means committing this patch or a work-alike change to pom.xml.

          Show
          Sean Owen added a comment - No, to be clear, this works for no situations at all – which includes the GroupLens example. The generated .war contains no Recommender that can be used with this package. If you've got a quick fix for the pom.xml that lets callers specify a .jar file, and include in the .war's WEB-INF/lib, that should fix all of this. I'd not spend too much time on this – we've lost a fair bit of time on this already. I need this mechanism to work, and have a working patch. I'd like it to work by the end of the week so we can move on at last with 0.1. I think that means committing this patch or a work-alike change to pom.xml.
          Hide
          Grant Ingersoll added a comment -

          If you've got a quick fix for the pom.xml that lets callers specify a .jar file, and include in the .war's WEB-INF/lib, that should fix all of this.

          I think I can do this, then by just adding a lib directory and having Maven look in there for any libs to include. Then, our instructions need only say: drop your Recommender jar in the lib directory, set the recommender.properties and then do "mvn package"

          I'll try it out this afternoon. If that works, then we can figure out the best way to pare down the WAR file.

          Show
          Grant Ingersoll added a comment - If you've got a quick fix for the pom.xml that lets callers specify a .jar file, and include in the .war's WEB-INF/lib, that should fix all of this. I think I can do this, then by just adding a lib directory and having Maven look in there for any libs to include. Then, our instructions need only say: drop your Recommender jar in the lib directory, set the recommender.properties and then do "mvn package" I'll try it out this afternoon. If that works, then we can figure out the best way to pare down the WAR file.
          Hide
          Grant Ingersoll added a comment -

          OK, I have the JAR stuff working.

          Sean, can you provide a list of the jars that shouldn't be in there? They are easy to exclude, too

          Also, from the looks of it, I can even have this just fire up Jetty.

          Basically, the instructions will be:
          Place your recommender in the lib directory
          mvn package

          For grouplens, a little more complex :
          copy grouplens Jar to the lib directory
          Place the data files in the appropriate place (if we do MAHOUT-94 it will make this easier)
          mvn package

          Show
          Grant Ingersoll added a comment - OK, I have the JAR stuff working. Sean, can you provide a list of the jars that shouldn't be in there? They are easy to exclude, too Also, from the looks of it, I can even have this just fire up Jetty. Basically, the instructions will be: Place your recommender in the lib directory mvn package For grouplens, a little more complex : copy grouplens Jar to the lib directory Place the data files in the appropriate place (if we do MAHOUT-94 it will make this easier) mvn package
          Hide
          Grant Ingersoll added a comment -

          OK, I think I've got it.

          this patch applies from the root of the trunk.

          Once applied do:

          • mvn install
          • cd taste-web
            *cp ../examples/target/grouplens.jar ./lib
          • Copy the group lens Data to:
            • I did:
            • mkdir -p ./src/main/resources/org/apache/mahout/cf/taste/example/grouplens
            • cp /Path/to/data/*.dat ./src/main/resources/org/apache/mahout/cf/taste/example/grouplens
          • Edit the recommender.properties file for your Recommender or just set -Drecommender.class on the command line when running the next line (note, the patch changes it to the GroupLens, but I would revert that)
          • mvn package //this step is likely optional, but I included it for completeness in case one doesn't want to run jetty
          • mvn jetty:run-war

          Point your browser at http://localhost:8080/mahout-taste-webapp/RecommenderServlet?userID=1

          NOTE, you may need to give the recommender more memory. To do this: export MAVEN_OPTS=-Xmx1024M (or whatever)

          Show
          Grant Ingersoll added a comment - OK, I think I've got it. this patch applies from the root of the trunk. Once applied do: mvn install cd taste-web *cp ../examples/target/grouplens.jar ./lib Copy the group lens Data to: I did: mkdir -p ./src/main/resources/org/apache/mahout/cf/taste/example/grouplens cp /Path/to/data/*.dat ./src/main/resources/org/apache/mahout/cf/taste/example/grouplens Edit the recommender.properties file for your Recommender or just set -Drecommender.class on the command line when running the next line (note, the patch changes it to the GroupLens, but I would revert that) mvn package //this step is likely optional, but I included it for completeness in case one doesn't want to run jetty mvn jetty:run-war Point your browser at http://localhost:8080/mahout-taste-webapp/RecommenderServlet?userID=1 NOTE, you may need to give the recommender more memory. To do this: export MAVEN_OPTS=-Xmx1024M (or whatever)
          Hide
          Grant Ingersoll added a comment -

          I'd say the only thing left to do is to put in the excludes for unneeded JARs. Personally, I'm not that concerned, b/c maybe it inspires some more cross-fertilization. I could see the web-app evolving to include a server for clustering, etc. Of course, I'm fine w/ doing the excludes, no need to guess on future functionality.

          Sean, does this work for you? I think the patch above keeps everything happy in Maven land and actually makes it a lot easier to run the demo, ala my goal in MAHOUT-94.

          If it does, I can commit and update the taste docs.

          Show
          Grant Ingersoll added a comment - I'd say the only thing left to do is to put in the excludes for unneeded JARs. Personally, I'm not that concerned, b/c maybe it inspires some more cross-fertilization. I could see the web-app evolving to include a server for clustering, etc. Of course, I'm fine w/ doing the excludes, no need to guess on future functionality. Sean, does this work for you? I think the patch above keeps everything happy in Maven land and actually makes it a lot easier to run the demo, ala my goal in MAHOUT-94 . If it does, I can commit and update the taste docs.
          Hide
          Sean Owen added a comment -

          The only .jars that should be needed are the two SLF4J jars, the Commons Logging jar, and Axis jars. I'd prefer to go ahead and exclude the unneeded parts.

          Where do I specify the name of the .jar to include that contains the Recommender implementation? It'd be OK to require it to have a certain name like "recommender.jar" and hard-code that it must be under lib, if that's better.

          I suppose I'd prefer the war be named "taste" as before, or "mahout-taste" just to make this path more manageable, but it's a tiny pointl.

          (I had just instructed people to copy dats to src/main/resources/org/apache/mahout/cf/taste/example/grouplens under the examples/ module and reference it there. Saves a "mkdir" and seems more appropriate.

          If mvn package produces a .war, great. The Jetty target is cool too.

          I can then swoop in and update the documentation and commit a few other tweaks I had in my patch. Then I think we're good to go as far as this issue is concerned.

          Show
          Sean Owen added a comment - The only .jars that should be needed are the two SLF4J jars, the Commons Logging jar, and Axis jars. I'd prefer to go ahead and exclude the unneeded parts. Where do I specify the name of the .jar to include that contains the Recommender implementation? It'd be OK to require it to have a certain name like "recommender.jar" and hard-code that it must be under lib, if that's better. I suppose I'd prefer the war be named "taste" as before, or "mahout-taste" just to make this path more manageable, but it's a tiny pointl. (I had just instructed people to copy dats to src/main/resources/org/apache/mahout/cf/taste/example/grouplens under the examples/ module and reference it there. Saves a "mkdir" and seems more appropriate. If mvn package produces a .war, great. The Jetty target is cool too. I can then swoop in and update the documentation and commit a few other tweaks I had in my patch. Then I think we're good to go as far as this issue is concerned.
          Hide
          Grant Ingersoll added a comment -

          The only .jars that should be needed are the two SLF4J jars, the Commons Logging jar, and Axis jars. I'd prefer to go ahead and exclude the unneeded parts.

          OK, I will update

          Where do I specify the name of the .jar to include that contains the Recommender implementation? It'd be OK to require it to have a certain name like "recommender.jar" and hard-code that it must be under lib, if that's better.

          Just drop the JAR file into the taste-web/lib directory, and it will automatically be added to the WAR file

          I suppose I'd prefer the war be named "taste" as before, or "mahout-taste" just to make this path more manageable, but it's a tiny pointl.

          Easy enough, I suspect.

          I can then swoop in and update the documentation and commit a few other tweaks I had in my patch. Then I think we're good to go as far as this issue is concerned.

          Glad to hear it. Once the docs are updated, I can do the release commands and we can have a vote. Yes!

          Show
          Grant Ingersoll added a comment - The only .jars that should be needed are the two SLF4J jars, the Commons Logging jar, and Axis jars. I'd prefer to go ahead and exclude the unneeded parts. OK, I will update Where do I specify the name of the .jar to include that contains the Recommender implementation? It'd be OK to require it to have a certain name like "recommender.jar" and hard-code that it must be under lib, if that's better. Just drop the JAR file into the taste-web/lib directory, and it will automatically be added to the WAR file I suppose I'd prefer the war be named "taste" as before, or "mahout-taste" just to make this path more manageable, but it's a tiny pointl. Easy enough, I suspect. I can then swoop in and update the documentation and commit a few other tweaks I had in my patch. Then I think we're good to go as far as this issue is concerned. Glad to hear it. Once the docs are updated, I can do the release commands and we can have a vote. Yes!
          Hide
          Grant Ingersoll added a comment -

          Note, also, the JAR file in the lib directory can be named whatever you want.

          Show
          Grant Ingersoll added a comment - Note, also, the JAR file in the lib directory can be named whatever you want.
          Hide
          Grant Ingersoll added a comment -

          OK, this one does the JAR excludes (it also deletes the mahout-assembly, but ignore that for now)

          Same directions as last time.

          Show
          Grant Ingersoll added a comment - OK, this one does the JAR excludes (it also deletes the mahout-assembly, but ignore that for now) Same directions as last time.
          Hide
          Grant Ingersoll added a comment -

          Here's a patch to taste.xml for the docs

          Show
          Grant Ingersoll added a comment - Here's a patch to taste.xml for the docs
          Hide
          Grant Ingersoll added a comment -

          We good to go on this one, Sean? I'd like to commit today.

          Show
          Grant Ingersoll added a comment - We good to go on this one, Sean? I'd like to commit today.
          Hide
          Sean Owen added a comment -

          I say go for it. I will merge with my patch locally and see if there is anything left and take care of that tonight. I'm glad this works out, many thanks.

          Show
          Sean Owen added a comment - I say go for it. I will merge with my patch locally and see if there is anything left and take care of that tonight. I'm glad this works out, many thanks.
          Hide
          Grant Ingersoll added a comment -

          Will do. Thanks for the kick in the pants to get going on it. I like the mvn jetty:run-war a lot. Now I can demo Taste next week!

          Show
          Grant Ingersoll added a comment - Will do. Thanks for the kick in the pants to get going on it. I like the mvn jetty:run-war a lot. Now I can demo Taste next week!
          Hide
          Grant Ingersoll added a comment -

          Committed

          Show
          Grant Ingersoll added a comment - Committed

            People

            • Assignee:
              Sean Owen
              Reporter:
              Sean Owen
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved:

                Development