Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-5884 Implement feature transformers to ML pipelines for Spark 1.4
  3. SPARK-5886

Add StringIndexer

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0
    • ML
    • None

    Description

      `StringIndexer` takes a column of string labels (raw categories) and outputs an integer column with labels indexed by their frequency.

      va li = new StringIndexer()
  .setInputCol("country")
  .setOutputCol("countryIndex")

      In the output column, we should store the label to index map as an ML attribute. The index should be ordered by frequency, where the most frequent label gets index 0, to enhance sparsity.

      We can discuss whether this should index multiple columns at the same time.

      Attachments

        Issue Links

          blocks

          Sub-task - The sub-task of the issue SPARK-6113 Stabilize DecisionTree and ensembles APIs

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is blocked by

          Sub-task - The sub-task of the issue SPARK-4588 Add API for feature attributes

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. SPARK-4081 Categorical feature indexing

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Documentation - Documentation or Website SPARK-7582 User guide update for StringIndexer

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #4735 [Github] Pull Request #4735 (mengxr)

          Activity

            People

              mengxr Xiangrui Meng
              mengxr Xiangrui Meng
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: