Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-1529

Finalize abstraction of distributed logical plans from backend operations

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: None
    • Labels:
      None

      Description

      We have a few situations when algorithm-facing API has Spark dependencies creeping in.

      In particular, we know of the following cases:
      (1) checkpoint() accepts Spark constant StorageLevel directly;
      (2) certain things in CheckpointedDRM;
      (3) drmParallelize etc. routines in the "drm" and "sparkbindings" package.

      (5) drmBroadcast returns a Spark-specific Broadcast object

      (6) Stratosphere/Flink conceptual api changes.

      Current tracker: PR #1 https://github.com/apache/mahout/pull/1 - closed, need new PR for remaining things once ready.
      Pull requests are welcome.

        Attachments

          Activity

            People

            • Assignee:
              dlyubimov Dmitriy Lyubimov
              Reporter:
              dlyubimov Dmitriy Lyubimov
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: