[FLINK-21111] FLIP-153: Support state access in Python DataStream API

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.13.0
• Component/s: API / Python
• Assignee: Wei Zhong
• Reporter: Shuiqiang Chen
• Created: 25/Jan/21 06:17
• Updated: 08/Apr/21 11:25
• Resolved: 08/Apr/21 11:25

Description

This is the umbrella Jira for FLIP-153, which intends to support state access in Python DataSteam API.

FLIP wiki page: https://cwiki.apache.org/confluence/display/FLINK/FLIP-153%3A+Support+state+access+in+Python+DataStream+API

In FLIP-130, we have already supported Python DataStream stateless API so that users are able to perform some basic data transformations. To implement more complex data processing, we need to provide state access support. In this doc, I would propose to add support of state access for Python DataStream API to support stateful operations on a KeyedStream.

Sub-Tasks

• Add ValueState/ListState/MapState and corresponding StateDescriptors for Python DataStream API: Closed
• Add State access API for KeyedStream RuntimeContext.: Closed
• Move the state module from 'pyflink.common' to 'pyflink.datastream': Closed
• Add ReducingState and corresponding StateDescriptor for Python DataStream API: Closed
• Add AggregatingState and corresponding StateDescriptor for Python DataStream API: Closed
• Make sure that the open/close methods of the Python DataStream Function are not implemented when using in ReducingState and AggregatingState: Open
• Add internal state hierarchy in PyFlink: Resolved
• Support setting namespace in RemoteKeyedStateBackend: Resolved
• Introduce InternalTimerService In PyFlink: Resolved
• Support state access API for the map/flat_map/filter/reduce operations of Python KeyedStream: Closed
• Support state access API for the map/flat_map operation of Python ConnectedStreams: Closed
• Add NamespacedStateView and PerWindowStateDataViewStore: Resolved