[SPARK-9847] ML Params copyValues should copy default values to default map, not set map

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: ML
• Target Version/s: 1.5.0
• Assignee: Joseph K. Bradley
• Reporter: Joseph K. Bradley
• Shepherd: Xiangrui Meng
• Created: 11/Aug/15 21:12
• Updated: 06/Nov/15 00:03
• Resolved: 12/Aug/15 17:50

Description

Currently, Params.copyValues copies default parameter values to the paramMap of the target instance, rather than the defaultParamMap. It should copy to the defaultParamMap because explicitly setting a parameter can change the semantics.

This issue arose in SPARK-9789, where 2 params "threshold" and "thresholds" for LogisticRegression can have mutually exclusive values. If thresholds is set, then fit() will copy the default value of threshold as well, easily resulting in inconsistent settings for the 2 params.

Issue Links

• blocks: SPARK-9789 Reinstate LogisticRegression threshold Param
• relates to: SPARK-11543 spark.ml LogisticRegressionModel should prefer thresholds over threshold
• links to: [Github] Pull Request #8115 (jkbradley)