[SPARK-5114] Should Evaluator be a PipelineStage

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 1.2.0
• Component/s: ML
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Joseph K. Bradley
• Created: 07/Jan/15 00:00
• Updated: 21/May/19 05:36
• Resolved: 21/May/19 05:36

Description

Pipelines can currently contain Estimators and Transformers.

Question for debate: Should Pipelines be able to contain Evaluators?

Pros:

• Schema check: Evaluators take input datasets with particular schema, which should perhaps be checked before running a Pipeline.
• Intermediate results:
  • If a Transformer removes a column (which is not done by built-in Transformers currently but might be reasonable in the future), then the user can never evaluate that column. (However, users could keep all columns around.)
  • If users have to evaluate after running a Pipeline, then each evaluated column may have to be re-materialized.

Cons:

• API: Evaluators do not transform datasets. They produce a scalar (or a few values), which makes it hard to say how they fit into a Pipeline or a PipelineModel.