Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-3376

Better expose BigQuery quota errors to users

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: io-java-gcp
    • Labels:
      None

      Description

      Currently users simply see following error.

      (9e02ab71e036fcc3): Workflow failed. Causes: (9e02ab71e036fbad): S27:Save to BigQuery/BatchLoads/MultiPartitionsReshuffle/GroupByKey/Read+Save to BigQuery/BatchLoads/MultiPartitionsReshuffle/GroupByKey/GroupByWindow+Save to BigQuery/BatchLoads/MultiPartitionsReshuffle/ExpandIterable+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/ParMultiDo(WriteTables)+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/WithKeys/AddKeys/Map+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/Window.Into()/Window.Assign+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/GroupByKey/Reify+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/GroupByKey/Write+Save to BigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/ParDo(ToIsmRecordForGlobalWindow) failed., (f88fcdac78f4b497): A work item was attempted 4 times without success. Each time the worker eventually lost contact with the service. The work item was attempted on: ...

      (there might be additional info in worker logs).

      We should update the BigQuery connector to raise an exception with a proper error message.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              chamikara Chamikara Jayalath
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: