Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Currently Kafka producer sends out messages asynchronous to different partitions, and there is no guarantee that all the messages will be persisted once watermark message is persisted. To avoid incorrect behavior, we need to put a produder.flush() before sending out the watermark message. This can be done in another thread to avoid the waiting for flush in process().