Mahout
  1. Mahout
  2. MAHOUT-1167

Parallel item similarity precomputation on a single machine

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8
    • Fix Version/s: 0.8
    • Labels:
      None

      Description

      We need some code for item-based CF usecases with an intermediate data size (e.g., a few million interactions). In such cases, the data might be too big to allow online computation of similarities and recommendations, but at the same time, going to Hadoop might still not be necessary and desired.

      In such a case, it makes sense to precompute item similarities on a single machine.

      1. MAHOUT-1167.patch
        26 kB
        Sebastian Schelter

        Activity

        Hide
        Sebastian Schelter added a comment -

        Patch for the parallel precomputation. Also introduces an example for the movielens1M dataset.

        Will commit this in two days if there are no objections.

        Show
        Sebastian Schelter added a comment - Patch for the parallel precomputation. Also introduces an example for the movielens1M dataset. Will commit this in two days if there are no objections.
        Hide
        Hudson added a comment -

        Integrated in Mahout-Quality #1912 (See https://builds.apache.org/job/Mahout-Quality/1912/)
        MAHOUT-1167 Parallel item similarity precomputation on a single machine (Revision 1458765)

        Result = FAILURE
        ssc : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458765
        Files :

        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/ItemSimilarityJob.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/SimilarItem.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute/FileSimilarItemsWriter.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute/MultithreadedBatchItemSimilarities.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/BatchItemSimilarities.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItem.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItems.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItemsWriter.java
        • /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity
        • /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute
        • /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/example
        • /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/example/BatchItemSimilaritiesGroupLens.java
        Show
        Hudson added a comment - Integrated in Mahout-Quality #1912 (See https://builds.apache.org/job/Mahout-Quality/1912/ ) MAHOUT-1167 Parallel item similarity precomputation on a single machine (Revision 1458765) Result = FAILURE ssc : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458765 Files : /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/ItemSimilarityJob.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/SimilarItem.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute/FileSimilarItemsWriter.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute/MultithreadedBatchItemSimilarities.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/BatchItemSimilarities.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItem.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItems.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItemsWriter.java /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/example /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/example/BatchItemSimilaritiesGroupLens.java

          People

          • Assignee:
            Sebastian Schelter
            Reporter:
            Sebastian Schelter
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development