Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22294

Reset spark.driver.bindAddress when starting a Checkpoint

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0, 2.2.0
    • Fix Version/s: 2.2.1, 2.3.0
    • Component/s: Deploy, Spark Core
    • Labels:
    • Target Version/s:
    • Flags:
      Patch

      Description

      On SPARK-4563 support for binding the driver to a different address than the spark.driver.host was provided so that the driver could be running under a differently-routed network. However, when the driver fails, the Checkpoint restoring function expects that the spark.driver.bindAddress remains the same even if the spark.driver.host variable may change. That limits the capabilities of recovery under several cluster configurations, and we propose that spark.driver.bindAddress should have the same replacement behaviour as spark.driver.host.

        Attachments

          Activity

            People

            • Assignee:
              ssaavedra Santiago Saavedra
              Reporter:
              ssaavedra Santiago Saavedra
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: