[SPARK-43981] Basic saving / loading implementation

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 3.5.0
• Component/s: Connect, ML, PySpark
• Assignee: Weichen Xu
• Reporter: Weichen Xu
• Created: 06/Jun/23 12:31
• Updated: 14/Jun/23 04:42
• Resolved: 14/Jun/23 04:42

Description

Support saving/loading  for estimator / transformer / evaluator / model.

We have some design goals:

• The model format is decoupled from spark, i.e. we can run model inference without spark service.
• We can save model to either local file system or cloud storage file system.