[FLINK-2513] Extend state handle provider interface to list all state handles

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Component/s: Runtime / Coordination
• Assignee: Unassigned
• Reporter: Ufuk Celebi
• Created: 13/Aug/15 11:55
• Updated: 20/Oct/15 12:55
• Resolved: 20/Oct/15 12:55

Description

This is a follow up issue to FLINK-2354.

In FLINK-2354 we use ZooKeeper to persist state handles. In certain failure scenarios, there can be lingering state handles, which have not been written to ZooKeeper, but which were already created.

These can be cleaned up on startup, if the state handle provider implementation kept track of their state handles. With the current implementations it would be possible, e.g. list the directory (file system) of no state is persistent (job manager).

It would be fair enough have this as an optional operation.