Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21221

CrossValidator and TrainValidationSplit Persist Nested Estimators such as OneVsRest

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.3.0
    • Component/s: ML
    • Labels:
      None

      Description

      Currently, the saving of parameters done in ValidatorParams.scala assumes that all parameters in EstimatorParameterMaps are JSON serializable. Such an assumption causes CrossValidator and TrainValidationSplit persistence to fail when the internal estimator to these meta-algorithms is not JSON serializable. One example is OneVsRest which has a classifier (estimator) as a parameter.

      The changes would involve removing the assumption in validator params that all the estimator params are JSON serializable. This could mean saving parameters that are not JSON serializable separately at a specified path.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ajaysaini Ajay Saini
                Reporter:
                ajaysaini Ajay Saini
                Shepherd:
                Joseph K. Bradley
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: