Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.14.0
-
None
-
None
Description
FLINK-20374 added an upsert materialization operator which fixes the order of shuffled streams. The results of this operator are actually eventually consistent (it collects the latest value it has seen and redacts older versions when these are not valid anymore). You could see a result stream like this, based on the order the materialization receives events:
+I10, -I10, +I5, -I5, +I10, -I10, +I3, -I3, +I10
Each time, the value stored in Kafka would change until the "final" result is in.
It may be acceptable for upsert sinks, but should be documented (or changed/fixed) nonetheless.