Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2691

Allow Spark on Mesos to be launched with Docker

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.4.0
    • Component/s: Mesos
    • Labels:
    • Target Version/s:

      Description

      Currently to launch Spark with Mesos one must upload a tarball and specifiy the executor URI to be passed in that is to be downloaded on each slave or even each execution depending coarse mode or not.

      We want to make Spark able to support launching Executors via a Docker image that utilizes the recent Docker and Mesos integration work.

      With the recent integration Spark can simply specify a Docker image and options that is needed and it should continue to work as-is.

        Attachments

        1. spark-docker.patch
          5 kB
          Chris Heller

          Issue Links

            Activity

              People

              • Assignee:
                ChrisHeller Chris Heller
                Reporter:
                tnachen Timothy Chen
              • Votes:
                9 Vote for this issue
                Watchers:
                24 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: