Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-10896 Extend state schema evolution support for more types
  3. FLINK-11333

First-class support for Protobuf types with evolvable schema

    XMLWordPrintableJSON

    Details

      Description

      I think we have more and more users who are thinking about using Protobuf for their state types.
      Right now, Protobuf isn't supported directly in Flink. The only way to use Protobuf for a type is to register it via Kryo: https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/custom_serializers.html.

      Likewise for Avro types, we should be able to natively support Protobuf, having a ProtobufSerializer that handles serialization of Protobuf types. The serializer should also write necessary information in its snapshot, to enable schema evolution for it in the future. For Protobuf, this should almost work out-of-the-box.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                yunta Yun Tang
                Reporter:
                tzulitai Tzu-Li (Gordon) Tai
              • Votes:
                2 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m