Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2691

Allow Spark on Mesos to be launched with Docker

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.0.0
    • 1.4.0
    • Mesos

    Description

      Currently to launch Spark with Mesos one must upload a tarball and specifiy the executor URI to be passed in that is to be downloaded on each slave or even each execution depending coarse mode or not.

      We want to make Spark able to support launching Executors via a Docker image that utilizes the recent Docker and Mesos integration work.

      With the recent integration Spark can simply specify a Docker image and options that is needed and it should continue to work as-is.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ChrisHeller Chris Heller
            tnachen Timothy Chen
            Votes:
            9 Vote for this issue
            Watchers:
            24 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment