Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8819

AvroCoder for SpecificRecords is not serialized correctly since 2.13.0

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: 2.13.0, 2.14.0, 2.15.0, 2.16.0
    • Fix Version/s: 2.17.0
    • Component/s: runner-core
    • Labels:
      None

      Description

      While trying to upgrade our pipelines from Beam 2.11.0 to Beam 2.16.0 we found that our SpecificRecords used in PCollection were being decoded as GenericRecords.

      After some investigation we found the specific commit/issue that we think did brake it:

      https://github.com/apache/beam/pull/8342/files

      https://issues.apache.org/jira/browse/BEAM-7103

      After the mentioned change all AvroCoders are serialized as simple urn: "beam:coder:avro:v1" which means they are deserialized / rehydrated as AvroCoder<GenericRecord>.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                piter Piotr Szczepanik
                Reporter:
                piter Piotr Szczepanik
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h
                  4h