This is an umbrella for discussing and creating ML abstractions. This was originally handled under
SPARK-1856 and SPARK-3702, under which we created the Pipelines API and some Developer APIs for classification and regression.
This umbrella is for future work, including:
- Stabilizing the classification and regression APIs
- Discussing traits vs. abstract classes for abstraction APIs
- Creating other abstractions not yet covered (clustering, multilabel prediction, etc.)
SPARK-3702 still has useful discussion and design docs.