[MAHOUT-1346] Spark Bindings (DRM)

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9
• Fix Version/s: 0.10.0
• Assignee: Dmitriy Lyubimov
• Reporter: Dmitriy Lyubimov
• Created: 07/Oct/13 20:50
• Updated: 13/Apr/15 10:20
• Resolved: 29/Apr/14 19:56

Description

Spark bindings for Mahout DRM.
DRM DSL.

Disclaimer. This will all be experimental at this point.

The idea is to wrap DRM by Spark RDD with support of some basic functionality, perhaps some humble beginning of Cost-based optimizer

(0) Spark serialization support for Vector, Matrix
(1) Bagel transposition
(2) slim X'X
(2a) not-so-slim X'X
(3) blockify() (compose RDD containing vertical blocks of original input)
(4) read/write Mahout DRM off HDFS
(5) A'B

...

Issue Links

• depends upon: MAHOUT-1297 New module for linear algebra scala DSL (in-core operators support only to start with)
• is depended upon by: MAHOUT-1365 Weighted ALS-WR iterator for Spark