Details
-
Bug
-
Status: Open
-
P3
-
Resolution: Unresolved
-
2.0.0
-
None
Description
When running a BigQueryIO query that doesn't return any rows (e.g. nothing has changed in a delta job) the job seems to stall and nothing happens as no temp files are being written which I think might be what it is waiting for. Just adding one row to the source table will make the job run through successfully.
Code:
PCollection <TableRow> rows = p.apply("ReadFromBQ", BigQueryIO.read() .fromQuery("SELECT * FROM `myproject.dataset.table`") .withoutResultFlattening().usingStandardSql());
Log:
Jun 02, 2017 9:00:36 AM org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob INFO: Started BigQuery job: {jobId=beam_job_batch-query, projectId=my-project}. bq show -j --format=prettyjson --project_id=my-project beam_job_batch-query Jun 02, 2017 9:03:11 AM org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract INFO: Starting BigQuery extract job: beam_job_batch-extract Jun 02, 2017 9:03:12 AM org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob INFO: Started BigQuery job: {jobId=beam_job_batch-extract, projectId=my-project}. bq show -j --format=prettyjson --project_id=my-project beam_job_batch-extract Jun 02, 2017 9:04:06 AM org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract INFO: BigQuery extract job completed: beam_job_batch-extract Jun 02, 2017 9:04:08 AM org.apache.beam.sdk.io.FileBasedSource expandFilePattern INFO: Matched 1 files for pattern gs://my-bucket/tmp/BigQueryExtractTemp/ff594d003c6440a1ad84b9e02858b5c6/000000000000.avro Jun 02, 2017 9:04:09 AM org.apache.beam.sdk.io.FileBasedSource getEstimatedSizeBytes INFO: Filepattern gs://my-bucket/tmp/BigQueryExtractTemp/ff594d003c6440a1ad84b9e02858b5c6/000000000000.avro matched 1 files with total size 9750