Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-732

Unable to upload AVRO file from GCS to BigQuery with no schema

    XMLWordPrintableJSON

    Details

      Description

      For the gcs_to_bq the operator you have to provide a schema and the fails if you do not provide a schema.

      However for AVRO uploads you do not need to provide a schema as this optional.

      Currently there is no schema is provided None is actually passed through which causes the job to fail for AVRO uploads.

      You can also actually tell the able to autodetect for CSV and JSON also using (configuration.load.autodetect) - more details https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tfayyaz Tahir Fayyaz
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: