[FLINK-4067] Add version header to savepoints

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.3
• Fix Version/s: 1.1.0
• Assignee: Ufuk Celebi
• Reporter: Ufuk Celebi
• Created: 13/Jun/16 15:15
• Updated: 26/Jul/16 09:31
• Resolved: 26/Jul/16 09:31

Description

Adding a header with version information to savepoints ensures that we can migrate savepoints between Flink versions in the future (for example when changing internal serialization formats between versions).

After talking with Till, we propose to add the following meta data:

• Magic number (int): identify data as savepoint
• Version (int): savepoint version (independent of Flink version)
• Data Offset (int): specifies at which point the actual savepoint data starts. With this, we can allow future Flink versions to add fields to the header without breaking stuff, e.g. Flink 1.1 could read savepoints of Flink 2.0.

For Flink 1.0 savepoint support, we have to try reading the savepoints without a header before failing if we don't find the magic number.

Issue Links

• links to: GitHub Pull Request #2194