Description
Copy GaussianMixture implementation from mllib to ml, then we can add new features to it.
I left mllib GaussianMixture untouched, unlike some other algorithms to wrap the ml implementation. For the following reasons:
- mllib GaussianMixture allow k == 1, but ml does not.
- mllib GaussianMixture supports setting initial model, but ml does not support currently. (We will definitely add this feature for ml in the future)
Meanwhile, There is a big performance improvement for GaussianMixture in this task. Since the covariance matrix of multivariate gaussian distribution is symmetric, we can only store the upper triangular part of the matrix and it will greatly reduce the shuffled data size.
Attachments
Issue Links
- is depended upon by
-
SPARK-17825 Expose log likelihood of EM algorithm in mllib
-
- Resolved
-
-
SPARK-14272 Evaluate GaussianMixtureModel with LogLikelihood
-
- Resolved
-
- links to
