Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-1503

Implement Nesterov's accelerated first-order method

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • None
    • None
    • MLlib

    Description

      Nesterov's accelerated first-order method is a drop-in replacement for steepest descent but it converges much faster. We should implement this method and compare its performance with existing algorithms, including SGD and L-BFGS.

      TFOCS (http://cvxr.com/tfocs/) is a reference implementation of Nesterov's method and its variants on composite objectives.

      Attachments

        1. linear.png
          46 kB
          Aaron Staple
        2. linear_l1.png
          39 kB
          Aaron Staple
        3. logistic.png
          39 kB
          Aaron Staple
        4. logistic_l2.png
          35 kB
          Aaron Staple

        Issue Links

          is part of

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-6346 Use faster converging optimization method in MLlib

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-3942 LogisticRegressionWithLBFGS should not use SquaredL2Updater

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-3382 GradientDescent convergence tolerance

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Pull request #4934 [Github] Pull Request #4934 (staple)

          Activity

            People

              staple Aaron Staple
              mengxr Xiangrui Meng
              Votes:
              1 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: