[SPARK-8536] Generalize LDA to asymmetric doc-topic priors

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: MLlib
• Target Version/s: 1.5.0
• Assignee: Feynman Liang
• Reporter: Joseph K. Bradley
• Shepherd: Joseph K. Bradley
• Created: 22/Jun/15 20:25
• Updated: 22/Jul/15 22:07
• Resolved: 22/Jul/15 22:07

Description

Several users have requested LDA using asymmetric priors, including learning those hyperparameters. We should add this functionality.

Note that it could be added to EM, online LDA, or both. I suggest adding it to online LDA initially, followed by EM in a separate PR.

This JIRA is now limited to the document-topic prior (docConcentration). See linked JIRA for topicConcentration, which is lower priority.

Issue Links

• is required by: SPARK-5572 LDA improvement listing
• relates to: SPARK-9134 LDA Asymmetric topic-word prior
• links to: [Github] Pull Request #7307 (feynmanliang)
• links to: [Github] Pull Request #7575 (feynmanliang)