Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2691

Allow Spark on Mesos to be launched with Docker

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.0.0
    • 1.4.0
    • Mesos

    Description

      Currently to launch Spark with Mesos one must upload a tarball and specifiy the executor URI to be passed in that is to be downloaded on each slave or even each execution depending coarse mode or not.

      We want to make Spark able to support launching Executors via a Docker image that utilizes the recent Docker and Mesos integration work.

      With the recent integration Spark can simply specify a Docker image and options that is needed and it should continue to work as-is.

      Attachments

        1. spark-docker.patch
          5 kB
          Chris Heller

        Issue Links

          Activity

            People

              ChrisHeller Chris Heller
              tnachen Timothy Chen
              Votes:
              9 Vote for this issue
              Watchers:
              24 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: