Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5894

HA docs are misleading re: state backends

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0, 1.2.1
    • Component/s: Documentation
    • Labels:
      None

      Description

      Towards the end of https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/jobmanager_high_availability.html#configuration it says "Currently, only the file system state backend is supported in HA mode."

      The state handles are written to the FileSystem and a reference to them is kept in ZooKeeper. So it's actually independent of the backend being used.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user uce opened a pull request:

          https://github.com/apache/flink/pull/3401

          FLINK-5894 [docs] Fix misleading HA docs

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/uce/flink 5894-ha_docs

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3401.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3401


          commit fa534a2a1f33a7d0d5d3665f90ea14cfff216557
          Author: Ufuk Celebi <uce@apache.org>
          Date: 2017-02-23T12:30:13Z

          FLINK-5894 [docs] Fix misleading HA docs


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user uce opened a pull request: https://github.com/apache/flink/pull/3401 FLINK-5894 [docs] Fix misleading HA docs You can merge this pull request into a Git repository by running: $ git pull https://github.com/uce/flink 5894-ha_docs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3401.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3401 commit fa534a2a1f33a7d0d5d3665f90ea14cfff216557 Author: Ufuk Celebi <uce@apache.org> Date: 2017-02-23T12:30:13Z FLINK-5894 [docs] Fix misleading HA docs
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alpinegizmo commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3401#discussion_r102703065

          — Diff: docs/setup/jobmanager_high_availability.md —
          @@ -84,12 +84,11 @@ In order to start an HA-cluster add the following configuration keys to `conf/fl

          *Important*: if you are running multiple Flink HA clusters, you have to manually configure separate namespaces for each cluster. By default, the Yarn cluster and the Yarn session automatically generate namespaces based on Yarn application id. A manual configuration overrides this behaviour in Yarn. Specifying a namespace with the -z CLI option, in turn, overrides manual configuration.

          – *State backend and storage directory* (required): JobManager meta data is persisted in the state backend and only a pointer to this state is stored in ZooKeeper. Currently, only the file system state backend is supported in HA mode.
          +- *Storage directory* (required): JobManager meta data is persisted in the file system storageDir and only a pointer to this state is stored in ZooKeeper.
          — End diff –

          It's generally agreed that metadata is one word.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alpinegizmo commented on a diff in the pull request: https://github.com/apache/flink/pull/3401#discussion_r102703065 — Diff: docs/setup/jobmanager_high_availability.md — @@ -84,12 +84,11 @@ In order to start an HA-cluster add the following configuration keys to `conf/fl * Important *: if you are running multiple Flink HA clusters, you have to manually configure separate namespaces for each cluster. By default, the Yarn cluster and the Yarn session automatically generate namespaces based on Yarn application id. A manual configuration overrides this behaviour in Yarn. Specifying a namespace with the -z CLI option, in turn, overrides manual configuration. – * State backend and storage directory * (required): JobManager meta data is persisted in the state backend and only a pointer to this state is stored in ZooKeeper. Currently, only the file system state backend is supported in HA mode. +- * Storage directory * (required): JobManager meta data is persisted in the file system storageDir and only a pointer to this state is stored in ZooKeeper. — End diff – It's generally agreed that metadata is one word.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alpinegizmo commented on the issue:

          https://github.com/apache/flink/pull/3401

          Would be better to spell metadata as one word. Otherwise looks great to me.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alpinegizmo commented on the issue: https://github.com/apache/flink/pull/3401 Would be better to spell metadata as one word. Otherwise looks great to me.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3401#discussion_r102705082

          — Diff: docs/setup/jobmanager_high_availability.md —
          @@ -84,12 +84,11 @@ In order to start an HA-cluster add the following configuration keys to `conf/fl

          *Important*: if you are running multiple Flink HA clusters, you have to manually configure separate namespaces for each cluster. By default, the Yarn cluster and the Yarn session automatically generate namespaces based on Yarn application id. A manual configuration overrides this behaviour in Yarn. Specifying a namespace with the -z CLI option, in turn, overrides manual configuration.

          – *State backend and storage directory* (required): JobManager meta data is persisted in the state backend and only a pointer to this state is stored in ZooKeeper. Currently, only the file system state backend is supported in HA mode.
          +- *Storage directory* (required): JobManager meta data is persisted in the file system storageDir and only a pointer to this state is stored in ZooKeeper.
          — End diff –

          Sure! I'm going to change and merge this.

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on a diff in the pull request: https://github.com/apache/flink/pull/3401#discussion_r102705082 — Diff: docs/setup/jobmanager_high_availability.md — @@ -84,12 +84,11 @@ In order to start an HA-cluster add the following configuration keys to `conf/fl * Important *: if you are running multiple Flink HA clusters, you have to manually configure separate namespaces for each cluster. By default, the Yarn cluster and the Yarn session automatically generate namespaces based on Yarn application id. A manual configuration overrides this behaviour in Yarn. Specifying a namespace with the -z CLI option, in turn, overrides manual configuration. – * State backend and storage directory * (required): JobManager meta data is persisted in the state backend and only a pointer to this state is stored in ZooKeeper. Currently, only the file system state backend is supported in HA mode. +- * Storage directory * (required): JobManager meta data is persisted in the file system storageDir and only a pointer to this state is stored in ZooKeeper. — End diff – Sure! I'm going to change and merge this.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3401

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3401
          Hide
          uce Ufuk Celebi added a comment -

          db3c5f3 (release-1.2), 234b905 (master).

          Show
          uce Ufuk Celebi added a comment - db3c5f3 (release-1.2), 234b905 (master).

            People

            • Assignee:
              Unassigned
              Reporter:
              alpinegizmo David Anderson
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development