Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-5564

Support sparse LDA solutions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 1.3.0
    • None
    • MLlib

    Description

      Latent Dirichlet Allocation (LDA) currently requires that the priors’ concentration parameters be > 1.0. It should support values > 0.0, which should encourage sparser topics (phi) and document-topic distributions (theta).

      For EM, this will require adding a projection to the M-step, as in: Vorontsov and Potapenko. "Tutorial on Probabilistic Topic Modeling : Additive Regularization for Stochastic Matrix Factorization." 2014.

      Attachments

        Issue Links

          is required by

          Umbrella - An overarching type made of sub-tasks SPARK-5572 LDA improvement listing

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              josephkb Joseph K. Bradley
              Votes:
              2 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: