Mahout
  1. Mahout
  2. MAHOUT-1167

Parallel item similarity precomputation on a single machine

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8
    • Fix Version/s: 0.8
    • Labels:
      None

      Description

      We need some code for item-based CF usecases with an intermediate data size (e.g., a few million interactions). In such cases, the data might be too big to allow online computation of similarities and recommendations, but at the same time, going to Hadoop might still not be necessary and desired.

      In such a case, it makes sense to precompute item similarities on a single machine.

      1. MAHOUT-1167.patch
        26 kB
        Sebastian Schelter

        Activity

        Sebastian Schelter created issue -
        Hide
        Sebastian Schelter added a comment -

        Patch for the parallel precomputation. Also introduces an example for the movielens1M dataset.

        Will commit this in two days if there are no objections.

        Show
        Sebastian Schelter added a comment - Patch for the parallel precomputation. Also introduces an example for the movielens1M dataset. Will commit this in two days if there are no objections.
        Sebastian Schelter made changes -
        Field Original Value New Value
        Attachment MAHOUT-1167.patch [ 12574134 ]
        Sebastian Schelter made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Sebastian Schelter made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Mahout-Quality #1912 (See https://builds.apache.org/job/Mahout-Quality/1912/)
        MAHOUT-1167 Parallel item similarity precomputation on a single machine (Revision 1458765)

        Result = FAILURE
        ssc : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458765
        Files :

        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/ItemSimilarityJob.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/SimilarItem.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute/FileSimilarItemsWriter.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute/MultithreadedBatchItemSimilarities.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/BatchItemSimilarities.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItem.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItems.java
        • /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItemsWriter.java
        • /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity
        • /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute
        • /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/example
        • /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/example/BatchItemSimilaritiesGroupLens.java
        Show
        Hudson added a comment - Integrated in Mahout-Quality #1912 (See https://builds.apache.org/job/Mahout-Quality/1912/ ) MAHOUT-1167 Parallel item similarity precomputation on a single machine (Revision 1458765) Result = FAILURE ssc : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458765 Files : /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/ItemSimilarityJob.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/SimilarItem.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute/FileSimilarItemsWriter.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/impl/similarity/precompute/MultithreadedBatchItemSimilarities.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/BatchItemSimilarities.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItem.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItems.java /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/SimilarItemsWriter.java /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/example /mahout/trunk/examples/src/main/java/org/apache/mahout/cf/taste/similarity/precompute/example/BatchItemSimilaritiesGroupLens.java
        Suneel Marthi made changes -
        Fix Version/s 0.8 [ 12320153 ]
        Suneel Marthi made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        1h 21m 1 Sebastian Schelter 18/Mar/13 12:04
        Patch Available Patch Available Resolved Resolved
        1d 23h 13m 1 Sebastian Schelter 20/Mar/13 11:18
        Resolved Resolved Closed Closed
        319d 20h 47m 1 Suneel Marthi 03/Feb/14 08:05

          People

          • Assignee:
            Sebastian Schelter
            Reporter:
            Sebastian Schelter
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development