[SPARK-4722] StreamingLinearRegression should return a DStream of weights when calling trainOn

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Component/s: DStreams, MLlib
• Labels: mllib regression streaming
• Assignee: Unassigned
• Reporter: Arthur Andres
• Shepherd: Xiangrui Meng
• Created: 03/Dec/14 11:42
• Updated: 06/Jan/16 10:06
• Resolved: 06/Jan/16 10:06

Description

When training a model with a stream of new data (Spark Streaming + Spark Mlllib), the weights (and the other part of the regression model) update at every iterations.
At the moment the only output we can get is the prediction when calling predictOn (class StreamingLinearRegression)
It would be a nice improvement if trainOn would return a Dstream of weights (and any other underlying model data) so we can access it and see it evolve. At the moment they are only outputted in the log
For example this could then be saved so when reloading the application we can access this information without having to train the model again.