[SPARK-20072] Clarify ALS-WR documentation

• Type: Improvement
• Status: Resolved
• Priority: Trivial
• Resolution: Incomplete
• Affects Version/s: 2.1.0
• Component/s: Documentation
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: chris snow
• Created: 23/Mar/17 14:17
• Updated: 21/May/19 04:11
• Resolved: 21/May/19 04:11

Description

https://www.mail-archive.com/[e-mail]/msg62590.html

The documentation for collaborative filtering is as follows:

===
Scaling of the regularization parameter

Since v1.1, we scale the regularization parameter lambda in solving
each least squares problem by the number of ratings the user generated
in updating user factors, or the number of ratings the product
received in updating product factors.
===

I find this description confusing, probably because I lack a detailed
understanding of ALS. The wording suggest that the number of ratings
change ("generated", "received") during solving the least squares.

This is how I think I should be interpreting the description:

===
Since v1.1, we scale the regularization parameter lambda when solving
each least squares problem. When updating the user factors, we scale
the regularization parameter by the total number of ratings from the
user. Similarly, when updating the product factors, we scale the
regularization parameter by the total number of ratings for the
product.
===