[SPARK-8555] Online Variational Inference for the Hierarchical Dirichlet Process

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Component/s: MLlib
• Assignee: Unassigned
• Reporter: yuhao yang
• Created: 23/Jun/15 06:16
• Updated: 07/Jan/16 11:20
• Resolved: 07/Jan/16 11:20

Description

The task is created for exploration on the online HDP algorithm described in
http://jmlr.csail.mit.edu/proceedings/papers/v15/wang11a/wang11a.pdf.

Major advantage for the algorithm: one pass on corpus, streaming friendly, automatic K (topic number).

Currently the scope is to support online HDP for topic modeling, i.e. probably an optimizer for LDA.

Issue Links

• is related to: SPARK-5572 LDA improvement listing