Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 1.0.0
    • Component/s: blueprints
    • Labels:
      None

      Description

      We should add an example for using Spark and MLlib to build an item recommender.

      Two challenges:
      1. The data generator does not generate user product ratings. We need a way to provide a metric for the "strength" of an interaction between a user and product. This could be the normalized purchase frequency for each product. Further evaluation is needed.

      2. How to evaluate the recommendations. We will want to divide the user data into 2 groups: validation and training. For the validation group, we may want to drop certain products and see if the recommender fills in those products or something similar.

        Issue Links

          Activity

          Hide
          jayunit100 jay vyas added a comment -

          BigTop BPS now supports ETL, SparkSQL, And ProductRecommendations for BPS at any scale. thanks RJ ! Commited.

          Show
          jayunit100 jay vyas added a comment - BigTop BPS now supports ETL, SparkSQL, And ProductRecommendations for BPS at any scale. thanks RJ ! Commited.
          Hide
          jayunit100 jay vyas added a comment -

          AWESOME. WE ARE NOW FEATURE COMPLETE with BIGPETSTORE-MAPREDUCE !

          Show
          jayunit100 jay vyas added a comment - AWESOME. WE ARE NOW FEATURE COMPLETE with BIGPETSTORE-MAPREDUCE !
          Hide
          jayunit100 jay vyas added a comment -

          we also should git commit --amend -m "[BigPetStore] Add BigPetStore Spark Product Recommender example" the commit message to match the new JIRA name (im appending [BigPetStore] to JIRAs so its clear what component these patches are focused on).

          Show
          jayunit100 jay vyas added a comment - we also should git commit --amend -m " [BigPetStore] Add BigPetStore Spark Product Recommender example" the commit message to match the new JIRA name (im appending [BigPetStore] to JIRAs so its clear what component these patches are focused on).
          Hide
          jayunit100 jay vyas added a comment -

          +1, as per conversation, i know this code is tested.

          This is awesome ! We will now have ETL, SparkSQL, Machine learning all implemented in BPS-Spark!

          Cant wait to scale test. I'll commit this when im at my apache box (or another commiter can commit for me).

          Thanks RJ

          Show
          jayunit100 jay vyas added a comment - +1, as per conversation, i know this code is tested. This is awesome ! We will now have ETL, SparkSQL, Machine learning all implemented in BPS-Spark! Cant wait to scale test. I'll commit this when im at my apache box (or another commiter can commit for me). Thanks RJ
          Hide
          rnowling RJ Nowling added a comment -

          jay vyas Can you review when you have time?

          Show
          rnowling RJ Nowling added a comment - jay vyas Can you review when you have time?
          Hide
          rnowling RJ Nowling added a comment -

          Adds an example for product recommendations including data models and JSON input/output functions. Added example to integration test. Updated README.

          Show
          rnowling RJ Nowling added a comment - Adds an example for product recommendations including data models and JSON input/output functions. Added example to integration test. Updated README.

            People

            • Assignee:
              rnowling RJ Nowling
              Reporter:
              rnowling RJ Nowling
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development