[FLINK-6537] Umbrella issue for fixes to incremental snapshots

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Component/s: Runtime / State Backends
• Assignee: Stefan Richter
• Reporter: Stefan Richter
• Created: 11/May/17 09:43
• Updated: 05/Mar/20 15:09
• Resolved: 05/Mar/20 15:09

Description

This issue tracks ongoing fixes in the incremental checkpointing feature for the 1.3 release.

Sub-Tasks

• JobID should not be part of the registration key to the SharedStateRegistry: Closed
• SharedStateRegistry is disposing state handles from main thread: Closed
• Duplicated registration of new shared state when checkpoint confirmations are still pending: Closed
• OperatorSubtaskState has empty implementations of (un)/registerSharedStates: Closed
• Proactively cleanup local FS for RocksDBKeyedStateBackend on startup: Closed
• Lack of synchronization on materializedSstFiles in RocksDBKEyedStateBackend: Closed
• Incremental snapshots in RocksDB hold lock during async file upload: Closed
• Add more thorough tests for RocksDB and incremental checkpointing: Closed
• Make incremental checkpoints externalizable: Closed
• Communicate last committed checkpoint through trigger checkpoint message: Closed
• Register with shared state registry before adding to CompletedCheckpointStore: Closed
• Ensure registration of shared state happens before externalizing a checkpoint: Closed