[SPARK-20234] Improve Framework for Basic ML Tests

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.1.0
• Component/s: ML, Tests
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Bryan Cutler
• Created: 06/Apr/17 00:10
• Updated: 21/May/19 04:15
• Resolved: 21/May/19 04:15

Description

Recently, I came across a few simple ML bugs that should have been caught by basic checks, but there was a lack of coverage because the existing checks are just tacked on to other tests and easy to forget to add to new test suites.

I think it's worth spending some time to making these checks more standardized and easy to bring into new test suites so they are not overlooked - possibly by using a trait.

The basic checks I'm referring to are:

• parameter checks
• uid checks
• model/estimator copies
• model/estimator persistence