Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7379

Flink runner build instructions need updating

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: website
    • Labels:
      None

      Description

      Hi,

      I'm new to the project and trying to follow the instructions on https://beam.apache.org/documentation/runners/flink/ for the portable flink runner, but they are out of date. I'd love to see the docs updated so that users like myself can do this successfully.

      The first issue is that all of the source releases that I downloaded, from both github and the downloads page, had the gradlew file removed.  I had to clone the repo to solve this problem.

      The next problem that I encountered was this:

      chad$ ./gradlew :sdks:python:container:docker
      Configuration on demand is an incubating feature.
      FAILURE: Build failed with an exception.
      * What went wrong:
      Project 'sdks' not found in root project 'beam'.
      

        

      The proper command is: 

      ./gradlew :beam-sdks-python-container:docker

       

      Likewise, the next command should be:

      ./gradlew :beam-runners-flink-1.7-job-server:runShadow -PflinkMasterUrl=localhost:8081

        

      After that I was able to get the flink beam example working.  I recommend adding this to the docs:

      (.venv) chad$ python -m apache_beam.examples.flink.flink_streaming_impulse

       

      also, since the docs specify only flink 1.7 is compatible with beam 2.12 (the latest as of this writing), I had to do a little extra work to get flink 1.7 built instead of 1.8, but this is probably outside the scope of the beam docs:

      brew install https://raw.github.com/Homebrew/homebrew-core/97b7a467beb81412bef2fbc497671c8ad3d407e5/Formula/apache-flink.rb
      /usr/local/Cellar/apache-flink/1.7.2/libexec/bin/start-cluster.sh

      thanks!

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              chadrik Chad Dombrova
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: