The broadcast operator state functionality was added in
FLINK-5265, it just hasn't been exposed through any public APIs yet.
Currently, we have 2 streaming connector features for 1.3 that are pending on broadcast state: rescalable Kinesis / Kafka consumers with shard / partition discovery (
FLINK-4821 & FLINK-4022). We should consider exposing broadcast state for the 1.3 release also.
This JIRA also serves the purpose to discuss how we want to expose it.
To initiate the discussion, I propose:
1. For the more powerful CheckpointedFunction, add the following to the OperatorStateStore interface:
2. For a simpler ListCheckpointed variant, we probably should have a separate BroadcastListCheckpointed interface.
Extending ListCheckpointed to let the user define either the list state type of either PARTITIONABLE or BROADCAST might also be possible, if we can rely on a contract that the value doesn't change. Or we expose a defining method (e.g. getListStateType()) that is called only once in the operator. This would break user code, but can be considered because it is marked as PublicEvolving.