[SPARK-16786] LDA topic distributions for new documents in PySpark

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 2.0.0
• Component/s: MLlib, PySpark
• Labels: patch

Environment

N/A

• Flags: Patch
• Assignee: Unassigned
• Reporter: Jordan Beauchamp
• Created: 28/Jul/16 22:32
• Updated: 04/Jan/17 00:13
• Resolved: 04/Jan/17 00:13

Description

pyspark.mllib.clustering.LDAModel has no way to estimate the topic distribution for new documents. However, this functionality exists in org.apache.spark.mllib.clustering.LDAModel. This change would only require setting up the API calls. I have forked the spark repo and implemented the changes locally

Issue Links

• depends upon: SPARK-5567 Add prediction methods to LDA
• links to: [Github] Pull Request #14394 (supremekai)