Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14808 Spark MLlib, GraphX, SparkR 2.0 QA umbrella
  3. SPARK-14813

ML 2.0 QA: API: Python API coverage

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Blocker
    • Resolution: Done
    • None
    • 2.0.0
    • Documentation, ML, PySpark
    • None

    Description

      For new public APIs added to MLlib, we need to check the generated HTML doc and compare the Scala & Python versions. We need to track:

      • Inconsistency: Do class/method/parameter names match?
      • Docs: Is the Python doc missing or just a stub? We want the Python doc to be as complete as the Scala doc.
      • API breaking changes: These should be very rare but are occasionally either necessary (intentional) or accidental. These must be recorded and added in the Migration Guide for this release.
        • Note: If the API change is for an Alpha/Experimental/DeveloperApi component, please note that as well.
      • Missing classes/methods/parameters: We should create to-do JIRAs for functionality missing from Python, to be added in the next release cycle. Please use a separate JIRA (linked below as "requires") for this list of to-do items.
        • NOTE: These missing features should be added in the next release. This work is just to generate a list of to-do items for the future.

      UPDATE: This only needs to cover spark.ml since spark.mllib is going into maintenance mode.

      Attachments

        Issue Links

          contains

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15623 2.0 python coverage ml.feature

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15628 pyspark.ml.evaluation module

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15630 2.0 python coverage ml root module

          • Major - Major loss of function.
          • Closed
          is cloned by

          Umbrella - An overarching type made of sub-tasks SPARK-16486 Python API parity issues from 2.0 QA

          • Major - Major loss of function.
          • Resolved
          requires

          Sub-task - The sub-task of the issue SPARK-11938 Expose numFeatures in all ML PredictionModel for PySpark

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-15316 PySpark GeneralizedLinearRegression missing linkPredictionCol param

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SPARK-8516 ML attribute API in PySpark

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SPARK-15181 Python API for Generalized Linear Regression Summary

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SPARK-14894 Python GaussianMixture summary

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-14978 PySpark TrainValidationSplitModel should support validationMetrics

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15402 PySpark ml.evaluation should support save/load

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15113 Add missing numFeatures & numClasses to wrapped JavaClassificationModel

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15130 PySpark shared params should include default values to match Scala

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15136 Linkify ML PyDoc

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15139 PySpark TreeEnsemble missing methods

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15194 Add Python ML API for MultivariateGaussian

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15442 PySpark QuantileDiscretizer missing "relativeError" param

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15106 Add package documentation for ML and remove BETA from Scala & Java for ML pipeline API.

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15162 Update PySpark LogisticRegression threshold PyDoc to be as complete as Scaladoc

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15163 Mark experimental algorithms experimental in PySpark

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15168 Add missing params to Python's MultilayerPerceptronClassifier

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15188 PySpark NaiveBayes is missing Thresholds param

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15189 ml.Evaluation pydoc issues

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15195 Improve PyDoc for ml.tuning

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15281 PySpark ML GBTRegressor lacks impurity param

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15412 Improve linear & isotonic regression methods PyDocs

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15788 PySpark IDFModel missing "idf" property

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Documentation - Documentation or Website SPARK-15500 Remove defaults in storage level param doc in ALS

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              holden Holden Karau
              josephkb Joseph K. Bradley
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: