[BEAM-2837] Writing To Spanner From Google Cloud DataFlow - Failure

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.1.0, 2.4.0
• Fix Version/s: 2.2.0
• Component/s: runner-dataflow
• Labels: stale-assigned
• Environment: Google Cloud DataFlow
• Assignee: Unassigned
• Reporter: Al Yaros
• Created: 02/Sep/17 19:06
• Updated: 24/Jul/20 20:17
• Resolved: 10/Jun/20 20:22

Description

Simple Java Program That reads from Pub\Sub and Writes to Spanner Fails with cryptic error message.

Simple Program to Demonstrate the Error:
https://github.com/alyaros/ExamplePubSubToSpannerViaDataFlow

*Caused by: org.apache.beam.sdk.util.UserCodeException: java.lang.NoClassDefFoundError: Could not initialize class com.google.cloud.spanner.spi.v1.SpannerErrorInterceptor
        org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
        org.apache.beam.sdk.io.
gcp.spanner.SpannerWriteGroupFn$DoFnInvoker.invokeSetup(Unknown Source)
        com.google.cloud.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.deserializeCopy(DoFnInstanceManagers.java:66)
        com.google.cloud.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.peek(DoFnInstanceManagers.java:48)
        com.google.cloud.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:104)
        com.google.cloud.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:66)
        com.google.cloud.dataflow.worker.MapTaskExecutorFactory.createParDoOperation(MapTaskExecutorFactory.java:360)
        com.google.cloud.dataflow.worker.MapTaskExecutorFactory$3.typedApply(MapTaskExecutorFactory.java:271)
        com.google.cloud.dataflow.worker.MapTaskExecutorFactory$3.typedApply(MapTaskExecutorFactory.java:253)
        com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:55)
        com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:43)
        com.google.cloud.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:78)
        com.google.cloud.dataflow.worker.MapTaskExecutorFactory.create(MapTaskExecutorFactory.java:142)
        com.google.cloud.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:925)
        com.google.cloud.dataflow.worker.StreamingDataflowWorker.access$800(StreamingDataflowWorker.java:133)
        com.google.cloud.dataflow.worker.StreamingDataflowWorker$7.run(StreamingDataflowWorker.java:771)
        java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        java.lang.Thread.run(Thread.java:745)*

Issue Links

• links to: GitHub Pull Request #3862
• links to: GitHub Pull Request #3907
• links to: GitHub Pull Request #3925