[BEAM-7531] Don't auto-create GCS buckets with KMS settings

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: Not applicable
• Component/s: io-java-gcp, io-py-gcp
• Assignee: Udi Meiri
• Reporter: Udi Meiri
• Created: 11/Jun/19 17:46
• Updated: 16/May/20 14:07
• Resolved: 21/Aug/19 22:16

Description

For Dataflow jobs, if a staging/temp path is not provided, a default bucket name is generated using the GCP project and job region.
https://github.com/apache/beam/blob/39dab79b7bd56a47c0f2a02033ab4ca3bd4a67e2/sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/options/GcpOptions.java#L319

We recently added the KMS setting to this bucket creation, but since different jobs might use different KMS keys this is unexpected behavior.

This bug is to remove this KMS setting from default bucket creation, and possibly error out when a default bucket is used and KMS settings are provided.

Issue Links

• links to: GitHub Pull Request #8830