Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11719

Enforce deterministic coding for GroupByKey and Stateful DoFns

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Triage Needed
    • P1
    • Resolution: Fixed
    • None
    • 2.29.0
    • sdk-py-core
    • None

    Description

      If a non-deterministic coder, such as pickling, is used for keys this can result in two copies of the same key being grouped separately (based on their encodings).

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. BEAM-11884 Deterministic coding enforcement causes BigQueryBatchFieldLoads/GroupFilesByTableDestinations to fail

          • P1 - Critical bug: data loss, total loss of function, or loss of testing signal due to test failures or flakiness
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. BEAM-12079 Deterministic coding enforcement causes _StreamToBigQuery/CommitInsertIds/GroupByKey to fail

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. BEAM-12552 GroupBy and GroupByKey doesn't group composite keys

          • P3 - Non-urgent bugs, features, and improvements
          • Resolved
          links to

          Web Link GitHub Pull Request #13839

          Web Link GitHub Pull Request #14180

          Web Link GitHub Pull Request #14680

          Activity

            People

              robertwb Robert Bradshaw
              robertwb Robert Bradshaw
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 14.5h
                  14.5h