[SPARK-10232] Decide whether spark.ml Decision Tree and Random Forest can replace spark.mllib implementation

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: ML, MLlib
• Assignee: Joseph K. Bradley
• Reporter: Joseph K. Bradley
• Created: 25/Aug/15 18:40
• Updated: 07/Dec/15 19:38
• Resolved: 07/Dec/15 19:33

Description

This JIRA is for discussing replacing the spark.mllib DecisionTree and RandomForest implementations with the implementation in spark.ml. The new implementation is simply a copy, with slight modifications (removing "bins").

Pros:

• Support only 1 implementation.
• Efficiency gains in spark.ml will benefit both APIs.

Cons:

• As spark.ml tree functionality increases, we will need to maintain conversion code for converting spark.ml trees to spark.mllib trees.

Must:

• Ensure we do not have significant regressions in the new implementation.

Issue Links

• blocks: SPARK-12183 Remove spark.mllib tree, forest implementations and use spark.ml
• relates to: SPARK-7131 Move tree,forest implementation from spark.mllib to spark.ml