Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Abandoned
-
None
-
None
Description
The proposal here is to enhance the way Queryable State works to allow for the state object to be transformed on the TaskManager before being returned to the client. As an example, if some MapState<K,V> were made queryable, such a transform might look up a specific key in the map and return its corresponding value, resulting in only that value being returned to the client instead of the entire map. This could be useful in cases where the client only wants a portion of the state and the state is large (this is my use case).
At a high level, I think this could be accomplished by adding an (optional) serializable Function<TState,TTransformed> into KvStateRequest (and related classes?) and having that transform be applied in the QueryableStateServer (or QueryableStateClientProxy?). I expect some additional TypeInformation would also have to be supplied/used in places. It should be doable in a backwards compatible way such that if the client does not specify a transform it works exactly as it does now.
Would there be any interested in a PR for this? This would help me for something I'm currently working on and I'd be willing to take a crack at it. If there is interest, I'll be happy to do some more research to come up with a more concrete proposal.
Thanks for Flink - it's great!