[BEAM-9268] SpannerIO: Better documentation and warning about creating tables in the pipeline

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.16.0, 2.17.0, 2.18.0, 2.19.0
• Fix Version/s: 2.20.0
• Component/s: io-go-gcp
• Labels: google-cloud-spanner perfomance
• Assignee: Niel Markwick
• Reporter: Niel Markwick
• Due: 03/Feb/20
• Created: 07/Feb/20 14:20
• Updated: 16/May/20 14:15
• Resolved: 10/Feb/20 23:00

Description

The javadoc for SpannerIO.Write mentions in passing that the transform needs to know the DB schema for optimal performance. If the schema is created within the pipeline, then there is a race between the schema being created and SpannerIO reading it, leading to a potential performance penalty if SpannerIO does not know about the existence of some tables. 

 

Javadoc needs to make this clearer and more explicit, and point the user at the Write.withSchemaReadySignal().

 

Pipeline needs to raise (rate limited) warnings if it sees writes being made to tables it does not know about (warnings can refer back to javadocs)

 

Issue Links

• links to: GitHub Pull Request #10690