[BEAM-2537] Can't run IO ITs - conflicting project vs projectId pipeline options

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Component/s: io-java-gcp
• Assignee: Stephen Sisk
• Reporter: Stephen Sisk
• Created: 28/Jun/17 21:12
• Updated: 24/Jul/20 20:19
• Resolved: 30/Jun/17 23:12

Description

Today, the GPC IO ITs use a mix of --project and --projectId pipeline options.

eg. GcpOptions.java uses --project:
https://github.com/apache/beam/blob/master/sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/options/GcpOptions.java#L81

and e.g the datastore IO ITs use that parameter:
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/datastore/V1ReadIT.java#L60

however, BigtableTestOptions uses --projectId:
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableTestOptions.java#L30

as does the spanner IO IT:
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/SpannerReadIT.java#L61

These conflicts mean that you can't run the tests as a suite successfully if you need to manually specify the project.

This works with the integration tests currently checked in b/c they are using the default project values (apache-beam-testing) when running in CI. However, that's not an option for running these tests as part of a suite when doing local development.

cc jasonkuster

Issue Links

• links to: GitHub Pull Request #3465