[BEAM-3589] Flink runner breaks with ClassCastException on UnboundedSource

• Type: Bug
• Status: Resolved
• Priority: P0
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.3.0
• Component/s: runner-flink
• Assignee: Grzegorz Kołakowski
• Reporter: Ismaël Mejía
• Created: 01/Feb/18 10:23
• Updated: 24/Jul/20 20:12
• Resolved: 06/Feb/18 15:55

Description

When you execute a pipeline tha uses an unbounded source and an empty transform it produces a ClassCastException:

[WARNING]
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:294)
    at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.ClassCastException: org.apache.beam.runners.core.construction.AutoValue_PTransformTranslation_UnknownRawPTransform cannot be cast to org.apache.beam.sdk.io.Read$Unbounded
    at org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$ReadSourceTranslator.translateNode (FlinkStreamingTransformTranslators.java:256)
    at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.applyStreamingTransform (FlinkStreamingPipelineTranslator.java:139)
    at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform (FlinkStreamingPipelineTranslator.java:118)
    at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:670)
    at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:647)
    at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:623)
    at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:623)
    at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:647)
    at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:662)
    at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600 (TransformHierarchy.java:311)
    at org.apache.beam.sdk.runners.TransformHierarchy.visit (TransformHierarchy.java:245)
    at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:458)
    at org.apache.beam.runners.flink.FlinkPipelineTranslator.translate (FlinkPipelineTranslator.java:38)
    at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate (FlinkStreamingPipelineTranslator.java:70)
    at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate (FlinkPipelineExecutionEnvironment.java:113)
    at org.apache.beam.runners.flink.FlinkRunner.run (FlinkRunner.java:110)
    at org.apache.beam.sdk.Pipeline.run (Pipeline.java:311)
    at org.apache.beam.sdk.Pipeline.run (Pipeline.java:297)
    at org.apache.beam.sdk.nexmark.NexmarkLauncher.run (NexmarkLauncher.java:1139)
    at org.apache.beam.sdk.nexmark.Main.runAll (Main.java:69)
    at org.apache.beam.sdk.nexmark.Main.main (Main.java:301)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:294)
    at java.lang.Thread.run (Thread.java:748)

You can reproduce it quickly by running this command from the nexmark directory:

mvn exec:java Dexec.mainClass=org.apache.beam.sdk.nexmark.Main -Pflink-runner "-Dexec.args=-runner=FlinkRunner --query=2 --streaming=true --manageResources=false --monitorJobs=true"

Issue Links

• relates to: BEAM-2806 support View.CreatePCollectionView in FlinkRunner