Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2490

Spark in HA when Zookeeper is available

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: spark
    • Labels:

      Description

      Spark could work in HA mode as long as a Zookeeper deployment is available. In this improvement we add an option to the Spark configuration through which we can set the Zookeeper ensemble string. This string is pushed down to Spark configs causing the Spark to function in HA mode.

      In HA mode Zookeeper is used to select the Spark leader and the workers need to ping all the potential leaders to pinpoint the current one.

      You can monitor the progress of this work in: https://github.com/juju-solutions/bigtop/tree/BIGTOP-2490

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ktsakalozos Konstantinos Tsakalozos
                Reporter:
                ktsakalozos Konstantinos Tsakalozos
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: