[SPARK-10354] First cost RDD shouldn't be cached in k-means|| and the following cost RDD should use MEMORY_AND_DISK

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.3.2, 1.4.2, 1.5.0
• Component/s: MLlib
• Assignee: Xiangrui Meng
• Reporter: Xiangrui Meng
• Created: 30/Aug/15 06:36
• Updated: 21/Dec/15 15:36
• Resolved: 31/Aug/15 06:22

Description

The first RDD doesn't need to be cached, other cost RDDs should use MEMORY_AND_DISK to avoid recomputing.

Issue Links

• is related to: SPARK-12450 Un-persist broadcasted variables in KMeans
• relates to: SPARK-10329 Cost RDD in k-means|| initialization is not storage-efficient
• links to: [Github] Pull Request #8526 (mengxr)