Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14772

Python ML Params.copy treats uid, paramMaps differently than Scala

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.1.1, 2.2.0
    • Component/s: ML, PySpark
    • Labels:
      None

      Description

      In PySpark, ml.param.Params.copy does not quite match the Scala implementation:

      • It does not copy the UID
      • It does not respect the difference between defaultParamMap and paramMap. This is an issue with _copyValues.

        Issue Links

          Activity

          Hide
          josephkb Joseph K. Bradley added a comment -

          Issue resolved by pull request 17048
          https://github.com/apache/spark/pull/17048

          Show
          josephkb Joseph K. Bradley added a comment - Issue resolved by pull request 17048 https://github.com/apache/spark/pull/17048
          Hide
          apachespark Apache Spark added a comment -

          User 'BryanCutler' has created a pull request for this issue:
          https://github.com/apache/spark/pull/17048

          Show
          apachespark Apache Spark added a comment - User 'BryanCutler' has created a pull request for this issue: https://github.com/apache/spark/pull/17048
          Hide
          apachespark Apache Spark added a comment -

          User 'BryanCutler' has created a pull request for this issue:
          https://github.com/apache/spark/pull/16772

          Show
          apachespark Apache Spark added a comment - User 'BryanCutler' has created a pull request for this issue: https://github.com/apache/spark/pull/16772
          Hide
          apachespark Apache Spark added a comment -

          User 'hujy' has created a pull request for this issue:
          https://github.com/apache/spark/pull/12888

          Show
          apachespark Apache Spark added a comment - User 'hujy' has created a pull request for this issue: https://github.com/apache/spark/pull/12888
          Hide
          hujiayin hujiayin added a comment - - edited

          @holdenk, I had a code for this issue and was busy with the other project in the past days. I just start to look into pyspark and look forward your comments.

          Show
          hujiayin hujiayin added a comment - - edited @holdenk, I had a code for this issue and was busy with the other project in the past days. I just start to look into pyspark and look forward your comments.
          Hide
          holdenk holdenk added a comment -

          I can take a look at this if no one else is working on it and it planned for 2.0

          Show
          holdenk holdenk added a comment - I can take a look at this if no one else is working on it and it planned for 2.0

            People

            • Assignee:
              bryanc Bryan Cutler
              Reporter:
              josephkb Joseph K. Bradley
              Shepherd:
              Joseph K. Bradley
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development