[FLINK-7720] Centralize creation of backends and state related resources

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.5.0
• Component/s: Runtime / State Backends
• Assignee: Stefan Richter
• Reporter: Stefan Richter
• Created: 27/Sep/17 15:43
• Updated: 25/Jan/18 11:06
• Resolved: 25/Jan/18 11:06

Description

State related resources like keyed state backends, operator state backends, timer service manager, etc. are currently all created in different places, somewhere between StreamTask and AbstractStreamOperator. It is hard to identify where/how resources are created, which component owns them, and how certain components depend on others (e.g. there is only a timer service manager if there is also a keyed state backend).

For the changes we plan for local state recovery, it would make sense to define a component that is responsible for the creation of state related resources and gives the possibility to interact in checkpointing and restoring with the checkpoint coordinator (owned by job manager) as well as local state storage (owned by task manager).

Issue Links

• is depended upon by: FLINK-8360 Implement task-local state recovery
• links to: GitHub Pull Request #4745