[BEAM-14283] Reduce number of threads spawned by BigqueryIO StreamingInserts

• Type: Improvement
• Status: Triage Needed
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.39.0
• Component/s: io-java-gcp
• Assignee: Arun Pandian
• Reporter: Arun Pandian
• Created: 11/Apr/22 07:07
• Updated: 13/Apr/23 11:09
• Resolved: 16/Apr/22 08:20

Description

BigqueryIO's BoundedExecutorService is creating many new threads. It can be optimized by acquiring the involved semaphore before task submission instead of after inside the runnable.

ref:https://github.com/apache/beam/blob/5603bd310a63fa35082e6bc6b06c7fecc90e77d7/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java#L1676

Issue Links

• links to: GitHub Pull Request #17333