[BEAM-3924] FlinkRunner fails to execute some of beam examples

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Affects Version/s: 2.4.0
• Component/s: runner-flink
• Assignee: Unassigned
• Reporter: Julien Tournay
• Created: 26/Mar/18 10:09
• Updated: 03/Jun/22 20:29

Description

I'm using:

• java version "1.8.0_162"
  Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
  Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
• Fink Version: 1.4.2, Commit ID: 04e4c85
• Beam 2.4.0

The examples are executed using a local Flink cluster as in Flink's Quickstart.
Most of the examples (WordCount, WindowedWordCount, etc.) are running fine.

The following example runs but thrown an exception:

mvn exec:java -Dexec.mainClass=org.apache.beam.examples.complete.TrafficRoutes \
    -Pflink-runner \
    -Dexec.args="--runner=FlinkRunner \
      --flinkMaster=localhost:6123 \
      --filesToStage=/private/tmp/word-count-beam/target/word-count-beam-bundled-0.1.jar \
      --tempLocation=gs://julient/temp \
      --bigQueryDataset=julient \
      --bigQueryTable=traffic_routes"

[WARNING]
java.lang.UnsupportedOperationException: FlinkRunnerResult does not support cancel.
    at org.apache.beam.runners.flink.FlinkRunnerResult.cancel (FlinkRunnerResult.java:64)
    at org.apache.beam.examples.common.ExampleUtils.lambda$addShutdownHook$0 (ExampleUtils.java:357)
    at java.lang.Thread.run (Thread.java:748)

Although an exception is thrown, the output BigQuery table is populated.

The following example runs but throws an exception and do not produce any output data:

mvn exec:java -Dexec.mainClass=org.apache.beam.examples.complete.TrafficMaxLaneFlow \
    -Pflink-runner \
    -Dexec.args="--runner=FlinkRunner \
      --inputFile=gs://apache-beam-samples/traffic_sensor/Freeways-5Minaa2010-01-01_to_2010-02-15_test2.csv \
      --flinkMaster=localhost:6123 \
      --filesToStage=target/word-count-beam-bundled-0.1.jar \
      --tempLocation=gs://julient/temp \
      --bigQueryDataset=julient \
      --bigQueryTable=traffic_max_lane_flow"

The exception is the same as the previous example:

[WARNING]
java.lang.UnsupportedOperationException: FlinkRunnerResult does not support cancel.
at org.apache.beam.runners.flink.FlinkRunnerResult.cancel (FlinkRunnerResult.java:64)
at org.apache.beam.examples.common.ExampleUtils.lambda$addShutdownHook$0 (ExampleUtils.java:357)
at java.lang.Thread.run (Thread.java:748)

 

Issue Links

• is blocked by: BEAM-593 Support unblocking run() in FlinkRunner and cancel() and waitUntilFinish() in FlinkRunnerResult