Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-13047

Pyspark Params.hasParam should not throw an error

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.1, 2.0.0
    • Component/s: ML, PySpark
    • Labels:
      None

      Description

      Pyspark Params class has a method hasParam(paramName) which returns True if the class has a parameter by that name, but throws an AttributeError otherwise. There is not currently a way of getting a Boolean to indicate if a class has a parameter. With Spark 2.0 we could modify the existing behavior of hasParam or add an additional method with this functionality.

      In Python:

      from pyspark.ml.classification import NaiveBayes
      nb = NaiveBayes(smoothing=0.5)
      print nb.hasParam("smoothing")
      print nb.hasParam("notAParam")
      

      produces:
      > True
      > AttributeError: 'NaiveBayes' object has no attribute 'notAParam'

      However, in Scala:

      import org.apache.spark.ml.classification.NaiveBayes
      val nb  = new NaiveBayes()
      nb.hasParam("smoothing")
      nb.hasParam("notAParam")
      

      produces:
      > true
      > false

        Attachments

          Activity

            People

            • Assignee:
              sethah Seth Hendrickson
              Reporter:
              sethah Seth Hendrickson
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: