[FLINK-17520] Extend CompositeTypeSerializerSnapshot to allow composite serializers to signal migration based on outer configuration

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.11.0, 1.12.0
• Component/s: API / Type Serialization System
• Labels: pull-request-available

Release Note

The `boolean isOuterSnapshotCompatible(TypeSerializer)` on the `CompositeTypeSerializerSnapshot` class has been deprecated, in favor of a new `OuterSchemaCompatibility resolveOuterSchemaCompatibility(TypeSerializer)` method. Please implement that instead. Compared to the old method, the new method allows composite serializers to signal state schema migration based on outer schema and configuration.

• Assignee: Tzu-Li (Gordon) Tai
• Reporter: Tzu-Li (Gordon) Tai
• Created: 05/May/20 08:48
• Updated: 16/Oct/20 10:50
• Resolved: 19/May/20 11:53

Description

Compatibility of composite serializers is governed by the overall resolved compatibility of all its nested serializers, as well as any additional configuration (or what we call the "outer configuration" or "outer snapshot").

The compatibility resolution logic for these composite serializers is implemented in the CompositeTypeSerializerSnapshot abstract class.

One current limitation of this base class is that the implementation assumes that the outer configuration is always either compatible, or incompatible.

We should relax this to also allow signaling migration, purely based on the outer configuration. This is already a requirement by FLINK-16998.

Issue Links

• blocks: FLINK-16998 Add a changeflag to Row type
• links to: GitHub Pull Request #12209