Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7803

BigQuery Streaming Inserts in Python perform GetTable call too many times

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • io-py-gcp
    • None

    Description

      Create table if needed is called once per destination per bundle. We should call it only once per destination at all. This can be done by passing the created tables as a side input.

       

      https://github.com/apache/beam/blob/v2.13.0/sdks/python/apache_beam/io/gcp/bigquery.py#L804-L816

      Attachments

        Activity

          People

            Unassigned Unassigned
            pabloem Pablo Estrada
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: