Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Component/s: Documentation
    • Labels:
      None

      Description

      EMR now has explicit Flink support, so there's no need to install Flink by hand.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user alpinegizmo opened a pull request:

          https://github.com/apache/flink/pull/3828

          FLINK-6447 update aws/emr docs

          EMR now has explicit Flink support, so there's no need to install Flink by hand unless you need something custom.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/alpinegizmo/flink 6447-update-aws-emr-docs

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3828.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3828


          commit 8e11b044c89eee0773e4bc4b5e1dff7e3223000e
          Author: David Anderson <david@alpinegizmo.com>
          Date: 2017-05-04T15:02:48Z

          FLINK-6447 update aws/emr docs


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user alpinegizmo opened a pull request: https://github.com/apache/flink/pull/3828 FLINK-6447 update aws/emr docs EMR now has explicit Flink support, so there's no need to install Flink by hand unless you need something custom. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alpinegizmo/flink 6447-update-aws-emr-docs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3828.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3828 commit 8e11b044c89eee0773e4bc4b5e1dff7e3223000e Author: David Anderson <david@alpinegizmo.com> Date: 2017-05-04T15:02:48Z FLINK-6447 update aws/emr docs
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user greghogan commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3828#discussion_r114861195

          — Diff: docs/setup/aws.md —
          @@ -32,17 +32,23 @@ Amazon Web Services offers cloud computing services on which you can run Flink.

          [Amazon Elastic MapReduce](https://aws.amazon.com/elasticmapreduce/) (Amazon EMR) is a web service that makes it easy to quickly setup a Hadoop cluster. This is the *recommended way* to run Flink on AWS as it takes care of setting up everything.

                1. Create EMR Cluster
                  +### Standard EMR Installation

          -The EMR documentation contains [examples showing how to start an EMR cluster](http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-gs-launch-sample-cluster.html). You can follow that guide and install any EMR release. You don't need to install All Applications part of the EMR release, but can stick to Core Hadoop:
          +Flink is a supported application on Amazon EMR. Basically all you have to do is choose Flink as an application, along with whatever
          +else you need, and configure the instances and roles. [Amazon's documentation](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-flink.html) gives all the details.

          -<img src="{{ site.baseurl }}/fig/flink-on-emr.png" class="img-responsive">
          +### Custom EMR Installation

          -When creating your cluster, make sure to setup [IAM roles](http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-iam-roles.html) allowing you to access your S3 buckets if required.
          +The standard installation (above) is easier, but if you need to use a version of Flink that Amazon doesn't support,
          — End diff –

          How about something like "Amazon EMR services are regularly updated to new releases but a version of Flink which is not available can be manually installed within a stock EMR cluster."?

          Show
          githubbot ASF GitHub Bot added a comment - Github user greghogan commented on a diff in the pull request: https://github.com/apache/flink/pull/3828#discussion_r114861195 — Diff: docs/setup/aws.md — @@ -32,17 +32,23 @@ Amazon Web Services offers cloud computing services on which you can run Flink. [Amazon Elastic MapReduce] ( https://aws.amazon.com/elasticmapreduce/ ) (Amazon EMR) is a web service that makes it easy to quickly setup a Hadoop cluster. This is the * recommended way * to run Flink on AWS as it takes care of setting up everything. Create EMR Cluster +### Standard EMR Installation -The EMR documentation contains [examples showing how to start an EMR cluster] ( http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-gs-launch-sample-cluster.html ). You can follow that guide and install any EMR release. You don't need to install All Applications part of the EMR release, but can stick to Core Hadoop : +Flink is a supported application on Amazon EMR. Basically all you have to do is choose Flink as an application, along with whatever +else you need, and configure the instances and roles. [Amazon's documentation] ( http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-flink.html ) gives all the details. -<img src="{{ site.baseurl }}/fig/flink-on-emr.png" class="img-responsive"> +### Custom EMR Installation -When creating your cluster, make sure to setup [IAM roles] ( http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-iam-roles.html ) allowing you to access your S3 buckets if required. +The standard installation (above) is easier, but if you need to use a version of Flink that Amazon doesn't support, — End diff – How about something like "Amazon EMR services are regularly updated to new releases but a version of Flink which is not available can be manually installed within a stock EMR cluster."?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user greghogan commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3828#discussion_r114827396

          — Diff: docs/setup/aws.md —
          @@ -32,17 +32,23 @@ Amazon Web Services offers cloud computing services on which you can run Flink.

          [Amazon Elastic MapReduce](https://aws.amazon.com/elasticmapreduce/) (Amazon EMR) is a web service that makes it easy to quickly setup a Hadoop cluster. This is the *recommended way* to run Flink on AWS as it takes care of setting up everything.

                1. Create EMR Cluster
                  +### Standard EMR Installation

          -The EMR documentation contains [examples showing how to start an EMR cluster](http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-gs-launch-sample-cluster.html). You can follow that guide and install any EMR release. You don't need to install All Applications part of the EMR release, but can stick to Core Hadoop:
          +Flink is a supported application on Amazon EMR. Basically all you have to do is choose Flink as an application, along with whatever
          +else you need, and configure the instances and roles. [Amazon's documentation](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-flink.html) gives all the details.

          -<img src="{{ site.baseurl }}/fig/flink-on-emr.png" class="img-responsive">
          — End diff –

          Can this png be deleted from the repo?

          Show
          githubbot ASF GitHub Bot added a comment - Github user greghogan commented on a diff in the pull request: https://github.com/apache/flink/pull/3828#discussion_r114827396 — Diff: docs/setup/aws.md — @@ -32,17 +32,23 @@ Amazon Web Services offers cloud computing services on which you can run Flink. [Amazon Elastic MapReduce] ( https://aws.amazon.com/elasticmapreduce/ ) (Amazon EMR) is a web service that makes it easy to quickly setup a Hadoop cluster. This is the * recommended way * to run Flink on AWS as it takes care of setting up everything. Create EMR Cluster +### Standard EMR Installation -The EMR documentation contains [examples showing how to start an EMR cluster] ( http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-gs-launch-sample-cluster.html ). You can follow that guide and install any EMR release. You don't need to install All Applications part of the EMR release, but can stick to Core Hadoop : +Flink is a supported application on Amazon EMR. Basically all you have to do is choose Flink as an application, along with whatever +else you need, and configure the instances and roles. [Amazon's documentation] ( http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-flink.html ) gives all the details. -<img src="{{ site.baseurl }}/fig/flink-on-emr.png" class="img-responsive"> — End diff – Can this png be deleted from the repo?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user greghogan commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3828#discussion_r114860033

          — Diff: docs/setup/aws.md —
          @@ -32,17 +32,23 @@ Amazon Web Services offers cloud computing services on which you can run Flink.

          [Amazon Elastic MapReduce](https://aws.amazon.com/elasticmapreduce/) (Amazon EMR) is a web service that makes it easy to quickly setup a Hadoop cluster. This is the *recommended way* to run Flink on AWS as it takes care of setting up everything.

                1. Create EMR Cluster
                  +### Standard EMR Installation

          -The EMR documentation contains [examples showing how to start an EMR cluster](http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-gs-launch-sample-cluster.html). You can follow that guide and install any EMR release. You don't need to install All Applications part of the EMR release, but can stick to Core Hadoop:
          +Flink is a supported application on Amazon EMR. Basically all you have to do is choose Flink as an application, along with whatever
          — End diff –

          How about something like "Flink is a supported application on Amazon EMR. [Amazon's documentation](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-flink.html) describes configuring Flink, creating and monitoring a cluster, and working with jobs."?

          Show
          githubbot ASF GitHub Bot added a comment - Github user greghogan commented on a diff in the pull request: https://github.com/apache/flink/pull/3828#discussion_r114860033 — Diff: docs/setup/aws.md — @@ -32,17 +32,23 @@ Amazon Web Services offers cloud computing services on which you can run Flink. [Amazon Elastic MapReduce] ( https://aws.amazon.com/elasticmapreduce/ ) (Amazon EMR) is a web service that makes it easy to quickly setup a Hadoop cluster. This is the * recommended way * to run Flink on AWS as it takes care of setting up everything. Create EMR Cluster +### Standard EMR Installation -The EMR documentation contains [examples showing how to start an EMR cluster] ( http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-gs-launch-sample-cluster.html ). You can follow that guide and install any EMR release. You don't need to install All Applications part of the EMR release, but can stick to Core Hadoop : +Flink is a supported application on Amazon EMR. Basically all you have to do is choose Flink as an application, along with whatever — End diff – How about something like "Flink is a supported application on Amazon EMR. [Amazon's documentation] ( http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-flink.html ) describes configuring Flink, creating and monitoring a cluster, and working with jobs."?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user greghogan commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3828#discussion_r114861184

          — Diff: docs/setup/aws.md —
          @@ -32,17 +32,23 @@ Amazon Web Services offers cloud computing services on which you can run Flink.

          [Amazon Elastic MapReduce](https://aws.amazon.com/elasticmapreduce/) (Amazon EMR) is a web service that makes it easy to quickly setup a Hadoop cluster. This is the *recommended way* to run Flink on AWS as it takes care of setting up everything.

                1. Create EMR Cluster
                  +### Standard EMR Installation

          -The EMR documentation contains [examples showing how to start an EMR cluster](http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-gs-launch-sample-cluster.html). You can follow that guide and install any EMR release. You don't need to install All Applications part of the EMR release, but can stick to Core Hadoop:
          +Flink is a supported application on Amazon EMR. Basically all you have to do is choose Flink as an application, along with whatever
          +else you need, and configure the instances and roles. [Amazon's documentation](http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-flink.html) gives all the details.

          -<img src="{{ site.baseurl }}/fig/flink-on-emr.png" class="img-responsive">
          +### Custom EMR Installation

          -When creating your cluster, make sure to setup [IAM roles](http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-iam-roles.html) allowing you to access your S3 buckets if required.
          +The standard installation (above) is easier, but if you need to use a version of Flink that Amazon doesn't support,
          +then you can setup a stock EMR cluster and install Flink yourself.

          -

          {% top %}

          +*Create EMR Cluster*
          +
          +The EMR documentation contains [examples showing how to start an EMR cluster](http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-gs-launch-sample-cluster.html). You can follow that guide and install any EMR release. You don't need to install the All Applications part of the EMR release, but can stick to Core Hadoop.
          +
          +When creating your cluster, make sure to setup [IAM roles](http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-iam-roles.html) allowing you to access your S3 buckets if required.
          — End diff –

          How about something like "When creating a cluster, access to S3 buckets requires configuration of [IAM roles](http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-iam-roles.html)."? And prefix with our "note" warning?

          Show
          githubbot ASF GitHub Bot added a comment - Github user greghogan commented on a diff in the pull request: https://github.com/apache/flink/pull/3828#discussion_r114861184 — Diff: docs/setup/aws.md — @@ -32,17 +32,23 @@ Amazon Web Services offers cloud computing services on which you can run Flink. [Amazon Elastic MapReduce] ( https://aws.amazon.com/elasticmapreduce/ ) (Amazon EMR) is a web service that makes it easy to quickly setup a Hadoop cluster. This is the * recommended way * to run Flink on AWS as it takes care of setting up everything. Create EMR Cluster +### Standard EMR Installation -The EMR documentation contains [examples showing how to start an EMR cluster] ( http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-gs-launch-sample-cluster.html ). You can follow that guide and install any EMR release. You don't need to install All Applications part of the EMR release, but can stick to Core Hadoop : +Flink is a supported application on Amazon EMR. Basically all you have to do is choose Flink as an application, along with whatever +else you need, and configure the instances and roles. [Amazon's documentation] ( http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-flink.html ) gives all the details. -<img src="{{ site.baseurl }}/fig/flink-on-emr.png" class="img-responsive"> +### Custom EMR Installation -When creating your cluster, make sure to setup [IAM roles] ( http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-iam-roles.html ) allowing you to access your S3 buckets if required. +The standard installation (above) is easier, but if you need to use a version of Flink that Amazon doesn't support, +then you can setup a stock EMR cluster and install Flink yourself. - {% top %} +* Create EMR Cluster * + +The EMR documentation contains [examples showing how to start an EMR cluster] ( http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-gs-launch-sample-cluster.html ). You can follow that guide and install any EMR release. You don't need to install the All Applications part of the EMR release, but can stick to Core Hadoop . + +When creating your cluster, make sure to setup [IAM roles] ( http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-iam-roles.html ) allowing you to access your S3 buckets if required. — End diff – How about something like "When creating a cluster, access to S3 buckets requires configuration of [IAM roles] ( http://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-iam-roles.html )."? And prefix with our "note" warning?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alpinegizmo commented on the issue:

          https://github.com/apache/flink/pull/3828

          @greghogan Thanks for the feedback!

          Show
          githubbot ASF GitHub Bot added a comment - Github user alpinegizmo commented on the issue: https://github.com/apache/flink/pull/3828 @greghogan Thanks for the feedback!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user StephanEwen commented on the issue:

          https://github.com/apache/flink/pull/3828

          +1 to merge this!

          Show
          githubbot ASF GitHub Bot added a comment - Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3828 +1 to merge this!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3828

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3828
          Hide
          StephanEwen Stephan Ewen added a comment -

          Fixed via 6c48f9bb0e27b86f57b940aac67db12c17b4f5bc

          Show
          StephanEwen Stephan Ewen added a comment - Fixed via 6c48f9bb0e27b86f57b940aac67db12c17b4f5bc

            People

            • Assignee:
              alpinegizmo David Anderson
              Reporter:
              alpinegizmo David Anderson
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development