Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2259

Spark submit documentation for --deploy-mode is highly misleading

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.0.1
    • Fix Version/s: 1.0.1, 1.1.0
    • Component/s: Documentation
    • Labels:
      None

      Description

      There are a few issues:

      1. Client mode does not necessarily mean the driver program must be launched outside of the cluster.

      2. For standalone clusters, only client mode is currently supported. This was the case supported even before 1.0.

      Currently, the docs tell the user to use cluster deploy mode when deploying "your driver program within the cluster," which is true also for standalone-client mode. In short, the docs encourage the user to use standalone-cluster, an unsupported mode.

        Attachments

          Activity

            People

            • Assignee:
              andrewor14 Andrew Or
              Reporter:
              andrewor14 Andrew Or
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: