[BEAM-5364] BigtableIO source tries to validate table ID even though validation is turned off

• Type: Bug
• Status: Resolved
• Priority: P3
• Resolution: Fixed
• Fix Version/s: Missing
• Component/s: io-java-gcp
• Assignee: Unassigned
• Reporter: Kevin Si
• Created: 11/Sep/18 20:46
• Updated: 13/Jan/22 21:52
• Resolved: 23/Jan/21 02:45

Description

https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java#L1084

The validation can be turned off with following:
BigtableIO.read()
            .withoutValidation() // skip validation when constructing the pipelline.

A Dataflow template cannot be constructed due to this validation failure.

 

Error log when trying to construct a template:

Exception in thread "main" java.lang.IllegalArgumentException: tableId was not supplied
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
        at org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$BigtableSource.validate(BigtableIO.java:1084)
        at org.apache.beam.sdk.io.Read$Bounded.expand(Read.java:95)
        at org.apache.beam.sdk.io.Read$Bounded.expand(Read.java:85)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:167)
        at org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$Read.expand(BigtableIO.java:423)
        at org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$Read.expand(BigtableIO.java:179)
        at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
        at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
        at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
        at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
        at com.google.cloud.teleport.bigtable.BigtableToAvro.main(BigtableToAvro.java:89)

Issue Links

• links to: GitHub Pull Request #6379
• links to: GitHub Pull Request #6413
• links to: GitHub Pull Request #6420