[SPARK-8519] Blockify distance computation in k-means

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 1.4.0
• Component/s: MLlib
• Labels: advanced bulk-closed
• Assignee: Yanbo Liang
• Reporter: Xiangrui Meng
• Shepherd: Xiangrui Meng
• Created: 22/Jun/15 06:21
• Updated: 16/Sep/19 20:28
• Resolved: 21/May/19 04:33

Description

The performance of pairwise distance computation in k-means can benefit from BLAS Level 3 matrix-matrix multiplications. It requires we update the implementation to use blocks. Even for sparse data, we might be able to see some performance gain.

Issue Links

• is related to: SPARK-11560 Optimize KMeans implementation / remove 'runs' from implementation
• links to: [Github] Pull Request #10306 (yanboliang)
• links to: [Github] Pull Request #10806 (yanboliang)
• links to: [Github] Pull Request #14937 (yanboliang)
• links to: GitHub Pull Request #14937