[HAMA-904] Fix Collaborative Filtering Example

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.6.4
• Fix Version/s: 0.8.0
• Component/s: examples, machine learning
• Labels: collaborative-filtering examples machine_learning
• Assignee: Unassigned
• Reporter: Martin Illecker
• Created: 19/May/14 13:48
• Updated: 15/Jun/15 03:43

Description

Fix Collaborative Filtering Example and revise test case.

I had a deep look into the collaborative filtering example of Ikhtiyor Ahmedov [1] and found the following questions / problems:

• Why are user and item features broadcasted [2] by each peer and not shared by a global input file on HDFS? (possible performance increase?)

• Use o.a.h.c.u.KeyValuePair instead of commons.math3.util.Pair in UserSimilarity [3] and ItemSimilarity [4]

• No need for normalizeWithBroadcastingValues [5] when taskNum = 1

• Why are values sent to itself and received later? [6] (possible performance increase?)

• Why is every task saving all items? [7] (duplicate saves?)

• Why is the default ALPHA value 0.01 and not 0.001? [8] (if (matrixRank > 250) -> Infinite / NaN)

I hope Ikhtiyor Ahmedov will finally become a committer and helps us to solve these questions.
Thanks!

[1] https://issues.apache.org/jira/browse/HAMA-612
[2] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/cf/OnlineTrainBSP.java#L116-128
[3] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/UserSimilarity.java#L22
[4] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/ItemSimilarity.java#L22
[5] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/cf/OnlineTrainBSP.java#L138
[6] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/cf/OnlineTrainBSP.java#L323
[7] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/cf/OnlineTrainBSP.java#L387-422
[8] https://github.com/apache/hama/blob/trunk/ml/src/main/java/org/apache/hama/ml/recommendation/cf/function/MeanAbsError.java#L62

Issue Links

• is related to: HAMA-612 BSP-based online collaborative filtering algorithm