[SPARK-15571] Pipeline unit test improvements

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Component/s: ML, PySpark
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Joseph K. Bradley
• Created: 26/May/16 18:05
• Updated: 21/May/19 04:14
• Resolved: 21/May/19 04:14

Description

Issue:

• There are several pieces of standard functionality shared by all algorithms: Params, UIDs, fit/transform/save/load, etc. Currently, these pieces are generally tested in ad hoc tests for each algorithm.
• This has led to inconsistent coverage, especially within the Python API.

Goal:

• Standardize unit tests for Scala and Python to improve and consolidate test coverage for Params, persistence, and other common functionality.
• Eliminate duplicate code. Improve test coverage. Simplify adding these standard unit tests for future algorithms and APIs.

This will require several subtasks. If you identify an issue, please create a subtask, or comment below if the issue needs to be discussed first.