Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-3433

Allow BigQueryIO to use a different project for the load job in batch mode.

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: io-java-gcp
    • Labels:
      None

      Description

      BigQueryIO is currently configured to always run a batch load job using the same projects as the destination table: https://github.com/apache/beam/blob/192b4c70927901860312f8c8acd27bd47e4a4259/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java#L256

      This may not always be desirable, since a pipeline may have write access to a dataset in a different project, but not jobs.create access in that project. This parameter should be settable in the interface.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                chamikara Chamikara Jayalath
                Reporter:
                kvncp Kevin Peterson
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 50m
                  1h 50m