[BEAM-11226] Support larger tables in BigQuery storage API bounded source

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.25.0
• Fix Version/s: 2.27.0
• Component/s: io-java-gcp
• Labels: stale-assigned
• Assignee: Kenneth Jung
• Reporter: Kenneth Jung
• Created: 10/Nov/20 20:55
• Updated: 14/Dec/20 23:48
• Resolved: 14/Dec/20 23:48

Description

The BigQuery storage API client specifies a default deadline of 10 minutes for calls to [CreateReadSession](https://cloud.google.com/bigquery/docs/reference/storage#create_a_session). However, for large tables, this operation can take longer than ten minutes, and as a result, read session creation simply fails for these tables. Some Beam runners, such as Google Cloud Dataflow, allow much longer for initial split calls on bounded sources – 2 hours, in the case of Dataflow. We can support larger BigQuery tables with the storage API source by raising the client-side deadline for CreateReadSession to match this 2-hour limit.

Issue Links

• links to: GitHub Pull Request #13295