Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21131

Fix batch gradient bug in SVDPlusPlus

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: 2.1.1
    • Fix Version/s: None
    • Component/s: GraphX
    • Labels:

      Description

      I think the mode of batch gradient in Graphx SVDPlusPlus is incorrect. The changes of this pull request are as blow:

      • Use the mean of batch gradient instead of the sum of batch gradient.
      • When calculating the residual of each rating during iteration, the pred value shouldn't be limit in (minVal, maxVal).
      • The method "calculateIterationRMSE" is added to show the iteration effect clearly.

      I test SVDPlusPlus locally by using movielens-100k dataset. The RMSE of origin SVDPlusPlus can't get converged and looks completely wrong. The RMSE of new SVDPlusPlus gets converged correctly.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              xmlyfly@gmail.com ruiqiangfan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: