Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-1856 Standardize MLlib interfaces
  3. SPARK-4588

Add API for feature attributes

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 1.4.0
    • ML, MLlib
    • None

    Description

      Feature attributes, e.g., continuous/categorical, feature names, feature dimension, number of categories, number of nonzeros (support) could be useful for ML algorithms.

      In SPARK-3569, we added metadata to schema, which can be used to store feature attributes along with the dataset. We need to provide a wrapper over the Metadata class for ML usage.

      The design doc is available at https://docs.google.com/document/d/1796XfSzFbZvGWFs0ky99AJhlqkOBRG1O2bUxK2N4Grk/edit?usp=sharing

      Attachments

        Issue Links

          blocks

          Sub-task - The sub-task of the issue SPARK-5886 Add StringIndexer

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-8515 Improve ML attribute API

          • Major - Major loss of function.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. SPARK-12886 Expose params to control how feature names are generated

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #4460 [Github] Pull Request #4460 (srowen)

          Pull request #4925 [Github] Pull Request #4925 (mengxr)

          Activity

            People

              srowen Sean R. Owen
              mengxr Xiangrui Meng
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: