Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4096

BigQueryIO ValueProvider support for NumFileShards and Triggering Frequency

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • 2.4.0
    • None
    • io-java-gcp
    • None

    Description

      Enhance BigQueryIO to accept ValueProviders for:

      • withTriggeringFrequency(..)
      • withNumFileShards(..)

      It would allow Dataflow templates to accept these parameters at runtime instead of being hardcoded. This opens up the ability to create Dataflow templates which allow users to flip back-and-forth between batch and streaming inserts.

      withMethod(..) cannot be changed at runtime currently.

      Attachments

        Activity

          People

            Unassigned Unassigned
            ryanmcdowell Ryan McDowell
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated: