Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
1.2.0
-
None
Description
Currently, the state interfaces (like ListState, ValueState, ReducingState) are very sparse and contain only methods exposed to the users. That makes sense to keep the public stable API minimal
At the same time, the runtime needs more methods for its internal interaction with state, such as:
- setting namespaces
- accessing raw values
- merging namespaces
These are currently realized by re-creating or re-obtaining the state objects from the KeyedStateBackend. That method causes quite an overhead for each access to the state
The KeyedStateBackend tries to do some tricks to reduce that overhead, but does it only partially and induces other overhead in the course.
The root cause of all these issues is a problem in the design: There is no proper "internal state abstraction" in a similar way as there is an external state abstraction (the public state API).
We should add a similar hierarchy of states for the internal methods. It would look like in the example below:
* State * | * +-------------------InternalKvState * | | * MergingState | * | | * +-----------------InternalMergingState * | | * +--------+------+ | * | | | * ReducingState ListState +-----+-----------------+ * | | | | * +-----------+ +----------- -----------------InternalListState * | | * +---------InternalReducingState
Attachments
Issue Links
- blocks
-
FLINK-5582 Add a general distributive aggregate function
-
- Closed
-