Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22294

Reset spark.driver.bindAddress when starting a Checkpoint

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete CommentsDelete
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.1.0, 2.2.0
    • 2.2.1, 2.3.0
    • Deploy, Spark Core
    • Patch

    Description

      On SPARK-4563 support for binding the driver to a different address than the spark.driver.host was provided so that the driver could be running under a differently-routed network. However, when the driver fails, the Checkpoint restoring function expects that the spark.driver.bindAddress remains the same even if the spark.driver.host variable may change. That limits the capabilities of recovery under several cluster configurations, and we propose that spark.driver.bindAddress should have the same replacement behaviour as spark.driver.host.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ssaavedra Santiago Saavedra Assign to me
            ssaavedra Santiago Saavedra
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment