Uploaded image for project: 'Apache Submarine'
  1. Apache Submarine
  2. SUBMARINE-570

Support run experiment/notebook with synced code

    XMLWordPrintableJSON

    Details

      Description

      In Submarine Experiment, there're needs to sync up code from Git/S3 to run experiment. 

      For example, Experiment can be defined to: 

      experiment: #Or notebook, model-serving
             name: "abc",
             environment: "team-default-ml-env"
             // Other fields
             code:
                 sync_mode: git
                 url: "https://foo.com/training-job.git"

      And once the experiment is launched, code from https://foo.com/training-job.git will be synced under /code directory of the launched container. 

      Different sync_mode can be supported, it can be git, s3, hdfs, nfs, etc.

      Similarily, notebook can sync code when a notebook session is launched, this avoids user manual checkout code post notebook session launch. 

      More design can be found at the following design docs:

      https://github.com/apache/submarine/blob/master/docs/design/experiment-implementation.md#api-of-experiment

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                manirajv06@gmail.com Manikandan R
                Reporter:
                wangda Wangda Tan
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: