Uploaded image for project: 'Hama'
  1. Hama
  2. HAMA-904

Fix Collaborative Filtering Example

    XMLWordPrintableJSON

    Details

      Description

      Fix Collaborative Filtering Example and revise test case.

      I had a deep look into the collaborative filtering example of Ikhtiyor Ahmedov [1] and found the following questions / problems:

      • Why are user and item features broadcasted [2] by each peer and not shared by a global input file on HDFS? (possible performance increase?)
      • Use o.a.h.c.u.KeyValuePair instead of commons.math3.util.Pair in UserSimilarity [3] and ItemSimilarity [4]
      • No need for normalizeWithBroadcastingValues [5] when taskNum = 1
      • Why are values sent to itself and received later? [6] (possible performance increase?)
      • Why is every task saving all items? [7] (duplicate saves?)
      • Why is the default ALPHA value 0.01 and not 0.001? [8] (if (matrixRank > 250) -> Infinite / NaN)

      I hope Ikhtiyor Ahmedov will finally become a committer and helps us to solve these questions.
      Thanks!

      [1] https://issues.apache.org/jira/browse/HAMA-612
      [2] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/cf/OnlineTrainBSP.java#L116-128
      [3] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/UserSimilarity.java#L22
      [4] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/ItemSimilarity.java#L22
      [5] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/cf/OnlineTrainBSP.java#L138
      [6] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/cf/OnlineTrainBSP.java#L323
      [7] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/cf/OnlineTrainBSP.java#L387-422
      [8] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/cf/function/MeanAbsError.java#L62

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                bafu Martin Illecker
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: