[SPARK-9690] Add random seed Param to PySpark CrossValidator

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.1
• Fix Version/s: 2.0.0
• Component/s: ML, PySpark
• Target Version/s: 2.0.0
• Assignee: Martin Menestret
• Reporter: Martin Menestret
• Shepherd: Joseph K. Bradley
• Created: 06/Aug/15 16:24
• Updated: 16/Dec/15 22:06
• Resolved: 16/Dec/15 22:06

Description

The fold in the ML CrossValidator depends on a rand whose seed is set to 0 and it leads the sql.functions rand to call sc._jvm.functions.rand() with no seed.

In order to be able to unit test a Cross Validation it would be a good idea to be able to set this seed so the output of the cross validation (with a featureSubsetStrategy set to "all") would always be the same.

Issue Links

• is related to: SPARK-9691 PySpark SQL rand function treats seed 0 as no seed
• links to: [Github] Pull Request #7997 (mmenestret)
• links to: [Github] Pull Request #10268 (jkbradley)