Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15243

Binarizer.explainParam(u"...") raises ValueError

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.3.0
    • Component/s: PySpark
    • Labels:
      None
    • Environment:

      CentOS 7, Spark 1.6.0

      Description

      When unicode is passed to Binarizer.explainParam(), ValueError occurs.

      To reproduce:

      >>> binarizer = Binarizer(threshold=1.0, inputCol="values", outputCol="features")
      >>> binarizer.explainParam("threshold") # str can be passed
      'threshold: threshold in binary classification prediction, in range [0, 1] (default: 0.0, current: 1.0)'
      
      >>> binarizer.explainParam(u"threshold") # unicode cannot be passed
      ---------------------------------------------------------------------------
      ValueError                                Traceback (most recent call last)
      <ipython-input-23-01c1345db0bd> in <module>()
      ----> 1 binarizer.explainParam(u"threshold")
      
      /usr/spark/current/python/pyspark/ml/param/__init__.py in explainParam(self, param)
           96         default value and user-supplied value in a string.
           97         """
      ---> 98         param = self._resolveParam(param)
           99         values = []
          100         if self.isDefined(param):
      
      /usr/spark/current/python/pyspark/ml/param/__init__.py in _resolveParam(self, param)
          231             return self.getParam(param)
          232         else:
      --> 233             raise ValueError("Cannot resolve %r as a param." % param)
          234 
          235     @staticmethod
      
      ValueError: Cannot resolve u'threshold' as a param.
      

      Same erros occur in other methods.

      • Binarizer.hasDefault()
      • Binarizer.getOrDefault()
      • Binarizer.isSet()

      These errors are caused by checks isinstance(obj, str) in pyspark.ml.param.Params._resolveParam().

      basestring should be used instead of str in isinstance() for backward compatibility as below.

      if sys.version >= '3':
           basestring = str
      
      if isinstance(obj, basestring):
          # TODO
      

        Attachments

          Activity

            People

            • Assignee:
              hyukjin.kwon Hyukjin Kwon
              Reporter:
              k-yokoshi Kazuki Yokoishi
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: