[FLINK-36318] Fail to restore from 1.18 if LAG function is used

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.19.0, 1.20.0, 1.19.1
• Fix Version/s: 1.19.2, 1.20.1, 2.0-preview
• Component/s: Table SQL / Planner
• Labels: pull-request-available
• Assignee: Dawid Wysakowicz
• Reporter: Dawid Wysakowicz
• Created: 18/Sep/24 15:42
• Updated: 20/Sep/24 19:53
• Resolved: 20/Sep/24 19:53

Description

One can not restore from a savepoint taken in 1.18 using Flink 1.19 if a query uses e.g. LAG function on a MAP type.

The reason is LAG/LEAD/ARRAY_AGG and possibly other functions use RAW type for accumulator.

In https://github.com/apache/flink/blob/d4c9ef165874e665b9e0f70cfe80fc3d387ac58e/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/typeutils/RowDataSerializer.java#L318 we deserialize/serialize types using java serialization which results in serializers being written into the snapshot.

In 1.19 https://github.com/apache/flink/blob/d4c9ef165874e665b9e0f70cfe80fc3d387ac58e/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/typeutils/MapDataSerializer.java#L54 was modified resulting in a change in serialVersionUID

As a result we fail to restore savepoints with a serialVersionUID mismatch.

Issue Links

• links to: GitHub Pull Request #25357
• links to: GitHub Pull Request #25358
• links to: GitHub Pull Request #25369