Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-7443

MLlib 1.4 QA plan

    XMLWordPrintableJSON

    Details

    • Type: Umbrella
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.4.0
    • Component/s: ML, MLlib
    • Labels:
      None
    • Target Version/s:

      Description

      API

      • Check API compliance using java-compliance-checker (SPARK-7458)
      • Audit new public APIs (from the generated html doc)
      • graduate spark.ml from alpha (SPARK-7748)
        • remove AlphaComponent annotations
        • remove mima excludes for spark.ml
        • mark concrete classes final wherever reasonable

      Algorithms and performance

      Performance

      Correctness

      Documentation and example code

      • Create JIRAs for the user guide to each new algorithm and assign them to the corresponding author. Link here as "requires"
        • Now that we have algorithms in spark.ml which are not in spark.mllib, we should start making subsections for the spark.ml API as needed. We can follow the structure of the spark.mllib user guide.
          • The spark.ml user guide can provide: (a) code examples and (b) info on algorithms which do not exist in spark.mllib.
          • We should not duplicate info in the spark.ml guides. Since spark.mllib is still the primary API, we should provide links to the corresponding algorithms in the spark.mllib user guide for more info.
      • Create example code for major components. Link here as "requires"
        • cross validation in python (SPARK-7387)
        • pipeline with complex feature transformations (scala/java/python) (SPARK-7546)
        • elastic-net (possibly with cross validation) (SPARK-7547)
        • kernel density (SPARK-7707)
      • Update Programming Guide for 1.4 (towards end of QA) (SPARK-7715)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                josephkb Joseph K. Bradley
                Reporter:
                mengxr Xiangrui Meng
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: