[SPARK-20861] Pyspark CrossValidator & TrainValidationSplit should delegate parameter looping to estimators

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.1
• Fix Version/s: 2.2.0, 2.3.0
• Component/s: ML, PySpark
• Target Version/s: 2.2.1, 2.3.0
• Assignee: Bago Amirbekian
• Reporter: Bago Amirbekian
• Created: 24/May/17 00:17
• Updated: 03/Jul/17 11:15
• Resolved: 24/May/17 03:56

Description

The CrossValidator & TrainValidationSplit should call estimator.fit with all their parameter maps instead of passing params one by one to fit. This behaviour would make Python spark more consistent with Scala spark and allow individual to parallelize or optimize for fitting over multiple parameter maps.

Issue Links

• links to: [Github] Pull Request #18077 (MrBago)