[SPARK-16365] Ideas for moving "mllib-local" forward

• Type: Brainstorming
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Component/s: ML
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Nicholas Pentreath
• Created: 04/Jul/16 13:29
• Updated: 21/May/19 04:16
• Resolved: 21/May/19 04:16

Description

Since SPARK-13944 is all done, we should all think about what the "next steps" might be for mllib-local. E.g., it could be "improve Spark's linear algebra", or "investigate how we will implement local models/pipelines in Spark", etc.

This ticket is for comments, ideas, brainstormings and PoCs. The separation of linalg into a standalone project turned out to be significantly more complex than originally expected. So I vote we devote sufficient discussion and time to planning out the next move

Issue Links

• relates to: SPARK-10413 ML models should support prediction on single instances