Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2495

Ability to re-create ML models

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.0.1
    • 1.1.0
    • MLlib
    • None

    Description

      Hi everyone.

      Previously (prior to Spark 1.0) we was working with MLib like this:

      1) Calculate model (costly operation)
      2) Take model and collect it's fields like weights, intercept e.t.c.
      3) Store model somewhere in our format
      4) Do predictions by loading model attributes, creating new model and predicting using it.

      Now i see that model's constructors have private modifier and cannot be created from outside.

      If you want to hide implementation details and keep this constructor as "developer api", why not to create at least method, which will take weights, intercept (for example) an materialize that model?

      A good example of model that i am talking about is: LinearRegressionModel

      I know that LinearRegressionWithSGD class have createModel method but the problem is that it have protected modifier as well.

      Attachments

        Issue Links

          Activity

            People

              gorenuru Alexander Albul
              gorenuru Alexander Albul
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: