[BEAM-2841] NPE in Dataflow job

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Duplicate
• Affects Version/s: 2.1.0
• Fix Version/s: 2.2.0
• Component/s: io-java-gcp
• Assignee: Chamikara Madhusanka Jayalath
• Reporter: Coda Hale
• Created: 05/Sep/17 18:15
• Updated: 24/Jul/20 20:17
• Resolved: 05/Sep/17 22:39

Description

My Dataflow job just started throwing exceptions after an unrelated update:

java.lang.NullPointerException org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:759) org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:809) org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.flushRows(StreamingWriteFn.java:126) org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.finishBundle(StreamingWriteFn.java:96)

The only change was to convert a DoFn<String, TableRow> to a DoFn<PubsubMessage, TableRow> and use PubsubIO.readMessages instead of PubsubIO.readString.

Any ideas?