[FLINK-3201] Enhance Partitioned State Interface with State Types

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.0.0
• Component/s: API / DataStream
• Assignee: Aljoscha Krettek
• Reporter: Aljoscha Krettek
• Created: 05/Jan/16 11:27
• Updated: 28/Feb/19 11:14
• Resolved: 08/Feb/16 15:20

Description

We should enhance the partitioned state with different state types, so that the system knows about the semantics of the State. I propose for now:

• ValueState, this behaves like the current OperatorState: state is one value that can be set and retrieved
• ListState, state is a list that can be appended to and iterated over
• ReducingState, state is one value that other values can be added to

ListState and ReducingState would share a common superclass to allow them to be used in the same places. For example, the WindowOperator would use ReducingState and ListState interchangeably, depending on whether we have a ReduceFunction or not.

These additions allow the system to be clever about how state is checkpointed in the future. Think ManageMemory/Out-of-core state and incremental checkpoints.

Also, state should be scoped to both a key and a namespace. This will allow the WindowOperator to use the interface. Right now, WindowOperator has a custom state implementation that uses a two-level Map (by key and by window). In this case the window would be the namespace.