Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-1542

Tutorial for playing with Mahout's Spark shell

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: Documentation, Math
    • Labels:
      None

      Description

      I have a created a tutorial for setting up the spark shell and implementing a simple linear regression algorithm. I'd love to make this part of the website, could someone give it a review?

      https://github.com/sscdotopen/krams/blob/master/linear-regression-cereals.md

      PS: If you wanna try out the code, you have to add the patch from MAHOUT-1532 to your sources.

        Activity

        Hide
        ssc Sebastian Schelter added a comment -

        Updated tutorial to also mention caching.

        Show
        ssc Sebastian Schelter added a comment - Updated tutorial to also mention caching.
        Hide
        Andrew_Palumbo Andrew Palumbo added a comment -

        Sebastian Schelter, I followed the tutorial step by step and everything worked without any issues. I found it very easy to follow. The cut and paste ols example worked for me easily. Very nice!

        Show
        Andrew_Palumbo Andrew Palumbo added a comment - Sebastian Schelter , I followed the tutorial step by step and everything worked without any issues. I found it very easy to follow. The cut and paste ols example worked for me easily. Very nice!
        Hide
        dlyubimov Dmitriy Lyubimov added a comment -

        This is super cool.

        One note i would add is that X is tall and skinny (so that X'X fits in memory but X is not). Otherwise it looks like the real thing all hapens in-core but this is not really the case.

        Show
        dlyubimov Dmitriy Lyubimov added a comment - This is super cool. One note i would add is that X is tall and skinny (so that X'X fits in memory but X is not). Otherwise it looks like the real thing all hapens in-core but this is not really the case.
        Hide
        dlyubimov Dmitriy Lyubimov added a comment -

        we also should probably modify shell so that this import is not needed.

        import org.apache.mahout.math.Vector
        
        Show
        dlyubimov Dmitriy Lyubimov added a comment - we also should probably modify shell so that this import is not needed. import org.apache.mahout.math.Vector
        Hide
        ssc Sebastian Schelter added a comment -

        added to the website. I also added a new top navigation point called "Spark". Shout if you don't like that naming.

        Show
        ssc Sebastian Schelter added a comment - added to the website. I also added a new top navigation point called "Spark". Shout if you don't like that naming.
        Hide
        dlyubimov Dmitriy Lyubimov added a comment -

        Sebastian Schelter do you mind if i rewrite the math symbols in latex/mathjax?

        Show
        dlyubimov Dmitriy Lyubimov added a comment - Sebastian Schelter do you mind if i rewrite the math symbols in latex/mathjax?
        Hide
        ssc Sebastian Schelter added a comment -

        No, go ahead, thats a great idea.

        Show
        ssc Sebastian Schelter added a comment - No, go ahead, thats a great idea.
        Hide
        dlyubimov Dmitriy Lyubimov added a comment -

        done in stage but for some reason it doesn't publish site for me. CMS infra problems again perhaps. Staging looks fine.

        Show
        dlyubimov Dmitriy Lyubimov added a comment - done in stage but for some reason it doesn't publish site for me. CMS infra problems again perhaps. Staging looks fine.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Mahout-Quality #2610 (See https://builds.apache.org/job/Mahout-Quality/2610/)
        MAHOUT-1542 Tutorial for playing with Mahout's Spark shell (ssc: rev 1595595)

        • /mahout/trunk/CHANGELOG
        • /mahout/trunk/spark-shell/src/main/scala/org/apache/mahout/sparkbindings/shell/MahoutSparkILoop.scala
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Mahout-Quality #2610 (See https://builds.apache.org/job/Mahout-Quality/2610/ ) MAHOUT-1542 Tutorial for playing with Mahout's Spark shell (ssc: rev 1595595) /mahout/trunk/CHANGELOG /mahout/trunk/spark-shell/src/main/scala/org/apache/mahout/sparkbindings/shell/MahoutSparkILoop.scala

          People

          • Assignee:
            ssc Sebastian Schelter
            Reporter:
            ssc Sebastian Schelter
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development