Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-5567

Add prediction methods to LDA

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.5.0
    • Component/s: MLlib
    • Labels:
      None
    • Target Version/s:

      Description

      LDA currently supports prediction on the training set. E.g., you can call logLikelihood and topicDistributions to get that info for the training data. However, it should support the same functionality for new (test) documents.

      This will require inference but should be able to use the same code, with a few modification to keep the inferred topics fixed.

      Note: The API for these methods is already in the code but is commented out.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                fliang Feynman Liang
                Reporter:
                josephkb Joseph K. Bradley
              • Votes:
                4 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 168h
                  168h
                  Remaining:
                  Remaining Estimate - 168h
                  168h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified