Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6192

Enhance MLlib's Python API (GSoC 2015)

    XMLWordPrintableJSON

Details

    Description

      This is an umbrella JIRA for MechCoder's GSoC 2015 project. The main theme is to enhance MLlib's Python API, to make it on par with the Scala/Java API. The main tasks are:

      1. For all models in MLlib, provide save/load method. This also
      includes save/load in Scala.
      2. Python API for evaluation metrics.
      3. Python API for streaming ML algorithms.
      4. Python API for distributed linear algebra.
      5. Simplify MLLibPythonAPI using DataFrames. Currently, we use
      customized serialization, making MLLibPythonAPI hard to maintain. It
      would be nice to use the DataFrames for serialization.

      I'll link the JIRAs for each of the tasks.

      Note that this doesn't mean all these JIRAs are pre-assigned to MechCoder. The TODO list will be dynamic based on the backlog.

      Attachments

        Issue Links

          contains

          Sub-task - The sub-task of the issue SPARK-4118 Create python bindings for Streaming KMeans

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SPARK-4127 Streaming Linear Regression- Python bindings

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SPARK-5989 Model import/export for LDAModel

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SPARK-7633 Streaming Logistic Regression- Python bindings

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue SPARK-7785 Add pretty printing to pyspark.mllib.linalg.Matrices

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-7844 Broken tests in KernelDensity

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-7946 DecayFactor wrongly set in StreamingKMeans

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-8032 Make NumPy version checking in mllib/__init__.py

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SPARK-6390 Add MatrixUDT in PySpark

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SPARK-7639 Add Python API for Statistics.kernelDensity

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-8265 Add LinearDataGenerator to pyspark.mllib.utils

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-8140 Remove empty model check in StreamingLinearAlgorithm

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-8291 Add parse functionality to LabeledPoint in PySpark

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is related to

          Brainstorming - A place to record back and forth on notions not yet formed enough to make a 'New Feature' or 'Task' SPARK-13489 GSoC 2016 project ideas for MLlib

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          relates to

          Umbrella - An overarching type made of sub-tasks SPARK-6254 MLlib Python API parity check at 1.3 release

          • Major - Major loss of function.
          • Closed

          Activity

            People

              MechCoder Manoj Kumar
              mengxr Xiangrui Meng
              Xiangrui Meng Xiangrui Meng
              Votes:
              2 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: