Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-7443

MLlib 1.4 QA plan

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.4.0
    • 1.4.0
    • ML, MLlib
    • None

    Description

      API

      • Check API compliance using java-compliance-checker (SPARK-7458)
      • Audit new public APIs (from the generated html doc)
      • graduate spark.ml from alpha (SPARK-7748)
        • remove AlphaComponent annotations
        • remove mima excludes for spark.ml
        • mark concrete classes final wherever reasonable

      Algorithms and performance

      Performance

      Correctness

      Documentation and example code

      • Create JIRAs for the user guide to each new algorithm and assign them to the corresponding author. Link here as "requires"
        • Now that we have algorithms in spark.ml which are not in spark.mllib, we should start making subsections for the spark.ml API as needed. We can follow the structure of the spark.mllib user guide.
          • The spark.ml user guide can provide: (a) code examples and (b) info on algorithms which do not exist in spark.mllib.
          • We should not duplicate info in the spark.ml guides. Since spark.mllib is still the primary API, we should provide links to the corresponding algorithms in the spark.mllib user guide for more info.
      • Create example code for major components. Link here as "requires"
        • cross validation in python (SPARK-7387)
        • pipeline with complex feature transformations (scala/java/python) (SPARK-7546)
        • elastic-net (possibly with cross validation) (SPARK-7547)
        • kernel density (SPARK-7707)
      • Update Programming Guide for 1.4 (towards end of QA) (SPARK-7715)

      Attachments

        Issue Links

          requires

          New Feature - A new feature of the product, which has yet to be developed. SPARK-7546 Example code for ML Pipelines feature transformations

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SPARK-7547 Example code for ElasticNet

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SPARK-7387 CrossValidator example code in Python

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-6013 Add more Python ML examples for spark.ml

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7272 User guide update for PMML model export

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7555 User guide update for ElasticNet

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7556 User guide update for feature transformer: Binarizer

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7557 User guide update for feature transformer: HashingTF, Tokenizer

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7574 User guide update for OneVsRest

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7575 Example code for OneVsRest

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7576 User guide update for spark.ml ElementwiseProduct

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7577 User guide update for Bucketizer

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7578 User guide update for spark.ml IDF, Normalizer, StandardScaler

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7579 User guide update for OneHotEncoder

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7581 User guide update for PolynomialExpansion

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7582 User guide update for StringIndexer

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7583 User guide update for RegexTokenizer

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7584 User guide update for VectorAssembler

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7585 User guide update for VectorIndexer

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7586 User guide update for spark.ml Word2Vec

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7707 User guide and example code for KernelDensity

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7772 User guide for spark.ml trees and ensembles

          • Major - Major loss of function.
          • Resolved

          Documentation - Documentation or Website SPARK-7459 Add Java example for ElementwiseProduct in programming guide

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Documentation - Documentation or Website SPARK-7496 User guide update for Online LDA

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          There are no Sub-Tasks for this issue.

          Activity

            People

              josephkb Joseph K. Bradley
              mengxr Xiangrui Meng
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: