[SPARK-14370] Avoid creating duplicate ids in OnlineLDAOptimizer

• Type: Improvement
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: MLlib
• Assignee: Pravin Gadakh
• Reporter: Joseph K. Bradley
• Created: 04/Apr/16 18:05
• Updated: 15/Apr/16 12:10
• Resolved: 15/Apr/16 12:08

Description

In OnlineLDAOptimizer's submitMiniBatch method, we create a list of ids val ids: List[Int] before calling variationalTopicInference, which then creates a duplicate set of ids. variationalTopicInference should reuse the same set.

Issue Links

• is required by: SPARK-5572 LDA improvement listing
• links to: [Github] Pull Request #12176 (pravingadakh)