[FLINK-3200] Use Partitioned State Abstraction in WindowOperator

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.0.0
• Component/s: API / DataStream
• Assignee: Aljoscha Krettek
• Reporter: Aljoscha Krettek
• Created: 05/Jan/16 11:18
• Updated: 28/Feb/19 11:14
• Resolved: 17/Feb/16 10:04

Description

Right now, the WindowOperator uses snapshotState and restoreState to do custom serialization/deserialization of the windowing state. Moving this to partitioned state would allow us to re-scale in the future. Also, once the partitioned state has support to run on ManagedMemory/Out-of-core state the WindowOperator would also automatically benefit from this, allowing very large windows and large window state.

The necessary steps are these:

• Enhance state interface with new primitive states: ReducableState, ListState, etc.. Also make state scoped to a namespace. Namespaces are necessary because we need state that is scoped by key and by window (window would be the namespace in this case)
• Enhance timer interface to also make timers scoped to a key/namespace
• Change WindowOperator to use these new interfaces

Sub-Tasks

• Enhance Partitioned State Interface with State Types: Resolved
• Make Timer/Trigger Service Scoped to Key/Namespace: Resolved
• Add convenience accessor methods for extended state interface: Closed