[BEAM-384] Streaming BigQueryIO should support user-provided IDs

• Type: Improvement
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Affects Version/s: 0.1.0-incubating, 0.2.0-incubating
• Component/s: io-java-gcp
• Assignee: Unassigned
• Reporter: Daniel Mills
• Created: 28/Jun/16 20:07
• Updated: 03/Jun/22 16:34

Description

Currently, BigQueryIO always assigns IDs and does a shuffle to ensure that they are atomic. This incurs a noticeable cost and is unnecessary if the user already has deterministic IDs that they can use. The sink should be able to use these IDs to skip the shuffle.