[SPARK-21729] Generic test for ProbabilisticClassifier to ensure consistent output columns

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.3.0
• Component/s: ML
• Assignee: Weichen Xu
• Reporter: Joseph K. Bradley
• Shepherd: Joseph K. Bradley
• Created: 14/Aug/17 22:32
• Updated: 02/Sep/17 00:32
• Resolved: 02/Sep/17 00:32

Description

One challenge with the ProbabilisticClassifier abstraction is that it introduces different code paths for predictions depending on which output columns are turned on or off: probability, rawPrediction, prediction. We ran into a bug in MLOR with this.

This task is for adding a generic test usable in all test suites for ProbabilisticClassifier types which does the following:

• Take a dataset + Estimator
• Fit the Estimator
• Test prediction using the model with all combinations of output columns turned on/off.
• Make sure the output column values match, presumably by comparing vs. the case with all 3 output columns turned on

CC WeichenXu123 since this came up in https://github.com/apache/spark/pull/17373

Issue Links

• relates to: SPARK-12664 Expose probability, rawPrediction in MultilayerPerceptronClassificationModel
• links to: [Github] Pull Request #19065 (WeichenXu123)