Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4096

BigQueryIO ValueProvider support for NumFileShards and Triggering Frequency

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • 2.4.0
    • None
    • io-java-gcp
    • None

    Description

      Enhance BigQueryIO to accept ValueProviders for:

      • withTriggeringFrequency(..)
      • withNumFileShards(..)

      It would allow Dataflow templates to accept these parameters at runtime instead of being hardcoded. This opens up the ability to create Dataflow templates which allow users to flip back-and-forth between batch and streaming inserts.

      withMethod(..) cannot be changed at runtime currently.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            ryanmcdowell Ryan McDowell

            Dates

              Created:
              Updated:

              Slack

                Issue deployment