[SPARK-11136] Warm-start support for ML estimator

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Incomplete
• Component/s: ML
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Xusen Yin
• Created: 15/Oct/15 21:53
• Updated: 08/Oct/19 05:43
• Resolved: 08/Oct/19 05:43

Description

The current implementation of Estimator does not support warm-start fitting, i.e. estimator.fit(data, params, partialModel). But first we need to add warm-start for all ML estimators. This is an umbrella JIRA to add support for the warm-start estimator.

Treat model as a special parameter, passing it through ParamMap. e.g. val partialModel: Param[Option[M]] = new Param(...). In the case of model existing, we use it to warm-start, else we start the training process from the beginning.

Issue Links

• contains: SPARK-7852 Set the initial weights based on the previous when GLMs are run with multiple regParams
• contains: SPARK-13856 Support initialModel in ALS
• contains: SPARK-15785 Add initialModel param to Gaussian Mixture Model (GMM) in spark.ml
• contains: SPARK-21386 ML LinearRegression supports warm start from user provided initial model
• contains: SPARK-10780 Set initialModel in KMeans in Pipelines API
• contains: SPARK-13025 Allow user to specify the initial model when training LogisticRegression
• is duplicated by: SPARK-13026 Umbrella: Allow user to specify initial model when training
• is related to: SPARK-20082 Incremental update of LDA model, by adding initialModel as start point
• relates to: SPARK-12098 Cross validator with multi-arm bandit search