[BEAM-8910] Use AVRO instead of JSON in BigQuery bounded source.

• Type: Improvement
• Status: Resolved
• Priority: P3
• Resolution: Fixed
• Fix Version/s: Missing
• Component/s: sdk-py-core
• Assignee: Pablo Estrada
• Reporter: Kamil Wasilewski
• Created: 06/Dec/19 15:14
• Updated: 13/Jan/22 21:52
• Resolved: 15/May/21 17:56

Description

The proposed BigQuery bounded source in Python SDK (see PR: https://github.com/apache/beam/pull/9772) uses a BigQuery export job to take a snapshot of the table and read from each produced JSON file. A performance improvement can be gain by switching to AVRO instead.

Issue Links

• is caused by: BEAM-1440 Create a BigQuery source (that implements iobase.BoundedSource) for Python SDK
• links to: GitHub Pull Request #11086