Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-3376

Better expose BigQuery quota errors to users

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • io-java-gcp
    • None

    Description

      Currently users simply see following error.

      (9e02ab71e036fcc3): Workflow failed. Causes: (9e02ab71e036fbad): S27:Save to BigQuery/BatchLoads/MultiPartitionsReshuffle/GroupByKey/Read+Save to BigQuery/BatchLoads/MultiPartitionsReshuffle/GroupByKey/GroupByWindow+Save to BigQuery/BatchLoads/MultiPartitionsReshuffle/ExpandIterable+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/ParMultiDo(WriteTables)+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/WithKeys/AddKeys/Map+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/Window.Into()/Window.Assign+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/GroupByKey/Reify+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/GroupByKey/Write+Save to BigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/ParDo(ToIsmRecordForGlobalWindow) failed., (f88fcdac78f4b497): A work item was attempted 4 times without success. Each time the worker eventually lost contact with the service. The work item was attempted on: ...

      (there might be additional info in worker logs).

      We should update the BigQuery connector to raise an exception with a proper error message.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              chamikara Chamikara Madhusanka Jayalath
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m