[BEAM-12079] Deterministic coding enforcement causes _StreamToBigQuery/CommitInsertIds/GroupByKey to fail

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.30.0
• Component/s: io-py-gcp
• Labels: stale-P2
• Assignee: Unassigned
• Reporter: Ludovic Post
• Created: 31/Mar/21 18:22
• Updated: 17/Aug/21 21:11
• Resolved: 01/Jun/21 03:27

Description

TypeError: Unable to deterministically encode '<TableReference
 datasetId: 'my-dataset-id'
 projectId: 'my-project-id'
 tableId: 'my-table-id'>' of type '<class 'apache_beam.io.gcp.internal.clients.bigquery.bigquery_v2_messages.TableReference'>', please provide a type hint for the input of 'WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/GroupByKey' [while running 'WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/Map(reify_timestamps)-ptransform-171']

 

Issue Links

• is caused by: BEAM-11719 Enforce deterministic coding for GroupByKey and Stateful DoFns
• is duplicated by: BEAM-12077 Deterministic coding enforcement causes _StreamToBigQuery/CommitInsertIds/GroupByKey to fail
• is duplicated by: BEAM-12078 Deterministic coding enforcement causes _StreamToBigQuery/CommitInsertIds/GroupByKey to fail
• relates to: BEAM-11884 Deterministic coding enforcement causes BigQueryBatchFieldLoads/GroupFilesByTableDestinations to fail
• relates to: BEAM-11408 GCP BigQuery sink (streaming inserts) uses runner determined sharding
• links to: GitHub Pull Request #14396
• links to: GitHub Pull Request #14415