[SPARK-21027] Parallel One vs. Rest Classifier

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0, 2.2.1
• Fix Version/s: 2.3.0
• Component/s: PySpark
• Assignee: Ajay Saini
• Reporter: Ajay Saini
• Shepherd: Joseph K. Bradley
• Created: 09/Jun/17 00:40
• Updated: 16/Oct/17 21:59
• Resolved: 12/Sep/17 17:03

Description

Currently, the Scala implementation of OneVsRest allows the user to run a parallel implementation in which each class is evaluated in a different thread. This implementation allows up to a 2X speedup as determined by experiments but is not currently not tunable. Furthermore, the python implementation of OneVsRest does not parallelize at all. It would be useful to add a parallel, tunable implementation of OneVsRest to the python library in order to speed up the algorithm.

A ticket for the Scala implementation of this classifier is here: https://issues.apache.org/jira/browse/SPARK-21028

Issue Links

• duplicates: SPARK-14450 Python OneVsRest should train multiple models at once
• is duplicated by: SPARK-21028 Parallel One vs. Rest Classifier Scala
• is related to: SPARK-19357 Parallel Model Evaluation for ML Tuning: Scala
• links to: [Github] Pull Request #18281 (ajaysaini725)
• links to: [Github] Pull Request #19110 (WeichenXu123)
• links to: [Github] Pull Request #19214 (WeichenXu123)