Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6763

Inefficient PojoSerializerConfigSnapshot serialization format

    XMLWordPrintableJSON

Details

    Description

      The PojoSerializerConfigSnapshot stores for each serializer the beginning offset and ending offset in the serialization stream. This information is also written if the serializer serialization is supposed to be ignored. The beginning and ending offsets are stored as a sequence of integers at the beginning of the serialization stream. We store this information to skip broken serializers.

      I think we don't need both offsets. Instead I would suggest to write the length of the serialized serializer first into the serialization stream and then the serialized serializer. This can be done in TypeSerializerSerializationUtil.writeSerializer. When reading the serializer via TypeSerializerSerializationUtil.tryReadSerializer, we can try to deserialize the serializer. If this operation fails, then we can skip the number of serialized serializer because we know how long it was.

      Attachments

        Issue Links

          Activity

            People

              tzulitai Tzu-Li (Gordon) Tai
              trohrmann Till Rohrmann
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: