Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18303

CLONE - Improving MLlib optimization APIs

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Major
    • Resolution: Invalid
    • None
    • None
    • MLlib
    • None

    Description

      Goal: Improve APIs for optimization

      Motivation: There have been several disjoint mentions of improving the optimization APIs to make them more pluggable, extensible, etc. This JIRA is a place to discuss what API changes are necessary for the long term, and to provide links to other relevant JIRAs.

      Eventually, I hope this leads to a design doc outlining:

      • current issues
      • requirements such as supporting many types of objective functions, optimization algorithms, and parameters to those algorithms
      • ideal API
      • breakdown of smaller JIRAs needed to achieve that API

      I will soon create an initial design doc, and I will try to watch this JIRA and include ideas from JIRA comments.

      Attachments

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-6849 The constructor of GradientDescent should be public

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-6682 Deprecate static train and use builder instead for Scala/Java

          • Major - Major loss of function.
          • Closed
          is a clone of

          Umbrella - An overarching type made of sub-tasks SPARK-5256 Improving MLlib optimization APIs

          • Major - Major loss of function.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. SPARK-2372 Grouped Optimization/Learning

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SPARK-2505 Weighted Regularizer

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-1486 Support multi-model training in MLlib

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-1227 Diagnostics for Classification&Regression

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-4526 Gradient should be added batch computing interface

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-5362 Gradient and Optimizer to support generic output (instead of label) and data batches

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-1457 Change APIs for training algorithms to take optimizer as parameter

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-11696 MLLIB:Optimization - Extend optimizer output for GradientDescent and LBFGS

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              iskenderunlu804 Iskender Unlu
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: