[SPARK-18693] BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.0.2
• Component/s: ML
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Devesh Parekh
• Created: 02/Dec/16 23:35
• Updated: 21/May/19 04:17
• Resolved: 21/May/19 04:17

Description

The LogisticRegression and LinearRegression models support training with a weight column, but the corresponding evaluators do not support computing metrics using those weights. This breaks model selection using CrossValidator.

Issue Links

• is a parent of: SPARK-24101 MulticlassClassificationEvaluator should use sample weight data
• is a parent of: SPARK-24102 RegressionEvaluator should use sample weight data
• is a parent of: SPARK-24103 BinaryClassificationEvaluator should use sample weight data
• relates to: SPARK-11520 RegressionMetrics should support instance weights
• links to: [Github] Pull Request #16557 (imatiach-msft)
• links to: [Github] Pull Request #17084 (imatiach-msft)
• links to: [Github] Pull Request #17085 (imatiach-msft)
• links to: [Github] Pull Request #17086 (imatiach-msft)