Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-11771

Serializer snapshot cannot be read if directly upgraded in-place to a TypeSerializerSnapshot from a TypeSerializerConfigSnapshot written in 1.7+

    XMLWordPrintableJSON

Details

    Description

      This is a upgrade path that was overlooked in TypeSerializerSnapshot#readVersionedSnapshot.

      If the serializer snapshot was a TypeSerializerConfigSnapshot when it was written in 1.7+ Flink versions, the actual snapshot content will be prefixed by a magic TypeSerializerConfigSnapshot#ADAPTER_VERSION int as the snapshot version, as well as Java-serialized prior serializer.

      If when restoring, the serializer snapshot was upgraded in-place to a TypeSerializerSnapshot (in-place meaning, same classname, not introducing a new snapshot class), TypeSerializerSnapshot#readVersionedSnapshot doesn't work as expected.
      Firstly, the provided readVersion to the user-implemented TypeSerializerSnapshot#readSnapshot method would be the magic ADAPTER_VERSION.
      Secondly, the remaining stream would contain the Java-serialized prior serializer, which is no longer relevant for the user (because they already upgraded to the new abstraction and would have a fully functional restoreSerializer implementation). To workaround that, they would have to read and drop that Java-serialized serializer in user code.

      Attachments

        Activity

          People

            tzulitai Tzu-Li (Gordon) Tai
            tzulitai Tzu-Li (Gordon) Tai
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: