[SPARK-9568] Spark MLlib 1.5.0 testing umbrella

• Type: Umbrella
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: MLlib
• Epic Link: Spark 1.5 Testing
• Assignee: Xiangrui Meng
• Reporter: Reynold Xin
• Created: 03/Aug/15 18:28
• Updated: 09/Nov/15 20:48
• Resolved: 11/Sep/15 07:34

Description

API

• Check binary API compatibility (SPARK-9658)
• Audit new public APIs (from the generated html doc)
  • Scala (SPARK-9660)
  • Java compatibility (SPARK-9661)
  • Python coverage (SPARK-9662)
• Check Experimental, DeveloperApi tags (SPARK-9665)

Algorithms and performance

Performance

• List any other missing performance tests from spark-perf here
• LDA online/EM (SPARK-7455)
• ElasticNet for linear regression and logistic regression (SPARK-7456)
• PIC (SPARK-7454)
• ALS.recommendAll (SPARK-7457)
• perf-tests in Python (SPARK-7539)

Correctness

• model save/load (SPARK-9666)

Documentation and example code

• For new algorithms, create JIRAs for updating the user guide (SPARK-9668)
• For major components, create JIRAs for example code (SPARK-9670)
• Update Programming Guide for 1.5 (towards end of QA) (SPARK-9671)
• Update website (SPARK-10120)

Issue Links

• is related to: SPARK-10120 Update MLlib website for 1.5
• relates to: SPARK-11600 Spark MLlib 1.6 QA umbrella
• requires: SPARK-7454 Perf test for power iteration clustering (PIC)
• requires: SPARK-7455 Perf test for LDA (EM/online)
• requires: SPARK-7456 Perf test for linear regression and logistic regression with elastic-net
• requires: SPARK-7457 Perf test for ALS.recommendProductsForUsers, recommendUsersForProducts
• requires: SPARK-7539 Perf tests for Python MLlib

Sub-Tasks

• ML 1.5 QA: API: Binary incompatible changes: Resolved
• ML 1.5 QA: API: New Scala APIs, docs: Resolved
• ML 1.5 QA: API: Java compatibility, docs: Resolved
• ML 1.5 QA: API: Python API coverage: Resolved
• ML 1.5 QA: API: Experimental, DeveloperApi, final, sealed audit: Resolved
• ML 1.5 QA: model save/load audit: Resolved
• ML 1.5 QA: Docs: Check for new APIs: Resolved
• ML 1.5 QA: Programming guide update and migration guide: Resolved