[SPARK-18316] Spark MLlib, GraphX 2.1 QA umbrella

• Type: Umbrella
• Status: Resolved
• Priority: Critical
• Resolution: Done
• Fix Version/s: 2.1.0
• Component/s: Documentation, GraphX, ML, MLlib
• Target Version/s: 2.1.0
• Assignee: Joseph K. Bradley
• Reporter: Joseph K. Bradley
• Created: 07/Nov/16 19:13
• Updated: 27/Apr/17 20:35
• Resolved: 09/Dec/16 21:14

Description

This JIRA lists tasks for the next Spark release's QA period for MLlib and GraphX. SparkR is separate: SPARK-18329.

The list below gives an overview of what is involved, and the corresponding JIRA issues are linked below that.

API

• Check binary API compatibility for Scala/Java
• Audit new public APIs (from the generated html doc)
  • Scala
  • Java compatibility
  • Python coverage
• Check Experimental, DeveloperApi tags

Algorithms and performance

• Performance tests
• Major new algorithms: MinHash, RandomProjection

Documentation and example code

• For new algorithms, create JIRAs for updating the user guide sections & examples
• Update Programming Guide
• Update website

Issue Links

• is a clone of: SPARK-14808 Spark MLlib, GraphX, SparkR 2.0 QA umbrella
• is cloned by: SPARK-20499 Spark MLlib, GraphX 2.2 QA umbrella
• is related to: SPARK-18329 Spark R 2.1 QA umbrella

Sub-Tasks

• ML, Graph 2.1 QA: API: Binary incompatible changes: Resolved
• ML, Graph 2.1 QA: API: New Scala APIs, docs: Resolved
• ML, Graph 2.1 QA: API: Experimental, DeveloperApi, final, sealed audit: Resolved
• ML 2.1 QA: API: Python API coverage: Resolved
• ML 2.1 QA: API: Java compatibility, docs: Resolved
• ML, Graph 2.1 QA: Update user guide for new features & APIs: Resolved
• ML, Graph 2.1 QA: Programming guide update and migration guide: Resolved
• Update MLlib, GraphX websites for 2.1: Resolved