Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4766

ML Estimator Params should be distinct from Transformer Params

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 1.2.0
    • Fix Version/s: None
    • Component/s: ML
    • Labels:
      None

      Description

      Currently, in spark.ml, both Transformers and Estimators extend the same Params classes. There should be one Params class for the Transformer and one for the Estimator. These could sometimes be the same, but for other models, we may need either (a) to make them distinct or (b) to have the Estimator params class extend the Transformer one.

      E.g., it is weird to be able to do:

      val model: LogisticRegressionModel = ...
      model.getMaxIter()
      

      It's also weird to be able to:

      • Wrap LogisticRegressionModel (a Transformer) with CrossValidator
      • Pass a set of ParamMaps to CrossValidator which includes parameter LogisticRegressionModel.maxIter
      • (CrossValidator would try to set that parameter.)
      • I'm not sure if this would cause a failure or just be a noop.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                josephkb Joseph K. Bradley
                Reporter:
                josephkb Joseph K. Bradley
              • Votes:
                1 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: