[SPARK-5017] GaussianMixtureEM should use SVD for Gaussian initialization

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.3.0
• Component/s: MLlib
• Target Version/s: 1.3.0
• Assignee: Travis Galoppo
• Reporter: Joseph K. Bradley
• Created: 30/Dec/14 20:42
• Updated: 28/Apr/15 20:59
• Resolved: 06/Jan/15 21:58

Description

GaussianMixtureEM effectively does 2 matrix decompositions in Gaussian initialization (pinv and det). Instead, it should do SVD and use that result to compute the inverse and det. This will also prevent failure when the matrix is singular.

Note: Breeze pinv fails when the matrix is singular: https://github.com/scalanlp/breeze/issues/304

Issue Links

• is required by: SPARK-7206 Gaussian Mixture Model (GMM) improvements
• links to: [Github] Pull Request #3871 (tgaloppo)