Uploaded image for project: 'SystemDS'
  1. SystemDS
  2. SYSTEMDS-1129

Enable parfor to run on remote Spark workers

    XMLWordPrintableJSON

Details

    • Epic
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Compiler, ParFor, Runtime
    • None
    • Enable parfor on remote Spark

    Description

      This issue tracks the development effort to enable parfor on Spark with parallel execution. It should be possible to force parfor to execute on distributed Spark workers in parallel.
      Execution should set up and remove local cache directories and respect custom configuration on remote workers.

      Attachments

        Issue Links

          is related to

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. SYSTEMDS-540 Deep Learning

          • Major - Major loss of function.
          • In Progress

          Improvement - An improvement or enhancement to an existing feature or task. SYSTEMDS-1159 Enable Remote Hyperparameter Tuning

          • Major - Major loss of function.
          • Open

          Activity

            People

              fschueler Felix Schueler
              fschueler Felix Schueler
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: