Details
-
Task
-
Status: Resolved
-
Blocker
-
Resolution: Unresolved
-
None
-
None
Description
When there are two sink workers running, there can be a case where one participant joins after the coordinator starts a first commit, which needs to be rolled back later since the other participant does not receive the START_COMMIT message for the transaction. In this case, later on in a new commit, `writeRecords()` can miss records because `ongoingTransactionInfo.getLastWrittenKafkaOffset()` is behind the record offsets in the buffer. This causes missing records in the target Hudi table.