[FLINK-21027] Add isStateKeyValueSerialized() method to KeyedStateBackend interface

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: Runtime / State Backends
• Labels: auto-deprioritized-major auto-unassigned pull-request-available
• Assignee: Yun Tang
• Reporter: Jark Wu
• Created: 19/Jan/21 10:38
• Updated: 09/Dec/21 07:35
• Resolved: 06/Dec/21 12:20

Description

In Table/SQL operators, we have some optimizations that reuse objects of keys and records. For example, we buffer input records in BytesMultiMap and use the reused object to map to the underlying memory segment to reduce bytes copy.

However, if we put the reused key and value into Heap statebackend, the result will be wrong, because it is not allowed to mutate keys and values in Heap statebackend.

Therefore, it would be great if KeyedStateBackend can expose such API, so that Table/SQL can dynamically decide whether to copy the keys and values before putting into state.

Issue Links

• relates to: FLINK-5151 Add discussion about object mutations to heap-based state backend docs.
• links to: GitHub Pull Request #17945