[BEAM-7506] Fake BigQuery JobService does not correctly handle nested fields

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: io-java-gcp
• Assignee: Unassigned
• Reporter: Charith Ellawala
• Created: 07/Jun/19 10:39
• Updated: 16/May/20 13:20

Description

writeRowsHelper method in the BigQuery FakeJobService class should take the data type of the field into consideration. For example, if a TableRow contains a RECORD field, it should be converted to an Avro GenericRecord first. Failing to do this results in a misleading null pointer exception thrown from the org.apache.beam.sdk.io.FileSystems.delete method even though the actual root cause for the failure is an Avro exception caused by the field type mismatch.