Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4253

Ignore spark.driver.host in yarn-cluster and standalone-cluster mode

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.2, 1.2.0
    • Component/s: YARN
    • Labels:
      None

      Description

      We actually don't know where driver will be before it is launched in yarn-cluster mode. If we set spark.driver.host property, Spark will create Actor on the hostname or ip as setted, which will leads an error.
      So we should ignore this config item in yarn-cluster mode.

      As Josh Rosen] pointed, we also ignore it in standalone cluster mode.

        Attachments

          Activity

            People

            • Assignee:
              WangTaoTheTonic Tao Wang (Inactive)
              Reporter:
              WangTaoTheTonic Tao Wang (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: