[FLINK-2324] Rework partitioned state storage

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Gyula Fora
• Reporter: Gyula Fora
• Created: 07/Jul/15 08:33
• Updated: 30/Jul/15 14:51
• Resolved: 30/Jul/15 14:51

Description

Partitioned states are currently stored per-key in statehandles. This is alright for in-memory storage but is very inefficient for HDFS.

The logic behind the current mechanism is that this approach provides a way to repartition a state without fetching the data from the external storage and only manipulating handles.

We should come up with a solution that can achieve both.