Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14324

Refactor GLMs code in SparkRWrappers

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • ML, SparkR
    • None

    Description

      We use a single object `SparkRWrappers` (https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/r/SparkRWrappers.scala) to wrap method calls to glm and kmeans in SparkR. This is quite hard to maintain. We should refactor them into separate wrappers, like `AFTSurvivalRegressionWrapper` and `NaiveBayesWrapper`.

      The package name should be `spakr.ml.r` instead of `spark.ml.api.r`.

      Attachments

        Issue Links

          blocks

          Sub-task - The sub-task of the issue SPARK-14315 GLMs model persistence in SparkR

          • Major - Major loss of function.
          • Resolved
          is a clone of

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-14303 Refactor k-means code in SparkRWrappers

          • Major - Major loss of function.
          • Resolved
          is blocked by

          New Feature - A new feature of the product, which has yet to be developed. SPARK-12566 GLM model family, link function support in SparkR:::glm

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Pull request #12294 [Github] Pull Request #12294 (yanboliang)

          Activity

            People

              yanboliang Yanbo Liang
              mengxr Xiangrui Meng
              Xiangrui Meng Xiangrui Meng
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: