[FLINK-19300] Timer loss after restoring from savepoint

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.8.0
• Fix Version/s: 1.11.3, 1.12.0
• Component/s: Runtime / State Backends
• Labels: pull-request-available
• Assignee: Xiang Gao
• Reporter: Xiang Gao
• Created: 21/Sep/20 04:09
• Updated: 08/Dec/21 09:03
• Resolved: 17/Nov/20 04:56

Description

While using heap-based timers, we are seeing occasional timer loss after restoring program from savepoint, especially when using a remote savepoint storage (s3). 

After some investigation, the issue seems to be related to this line in deserialization. When trying to check the VERSIONED_IDENTIFIER, the input stream may not guarantee filling the byte array, causing timers to be dropped for the affected key group.

Should keep reading until expected number of bytes are actually read or if end of the stream has been reached. 

Issue Links

• fixes: FLINK-10297 PostVersionedIOReadableWritable ignores result of InputStream.read(...)
• links to: GitHub Pull Request #14042