Major Description
When a record triggers a RetriableException and the retry is processed successfully, its offset does not commit.
The processing process of connect is as follows:
1: commitOffsets
2: poll (long timeoutMs)
3: convertMessages
4: deliverMessages
Offset storage:
1: lastCommittedOffsets
2: currentOffsets
3: origOffsets
Cause of the problem: I need to retry the record
1.RetriableException from SinkTask:
2.Pausing partitions
3.Not returning fetched records for assigned partition since it is no longer fetchable
3.1.ConsumerRecords<byte[], byte[]> msgs = pollConsumer(timeoutMs);
3.2.convertMessages(msgs);// msgs is empty
3.3.origOffsets.clear(); // Record of the retry operation, it's offset has been cleared
4.The retry operation has completed
4.1.now commit the offset of the record of the retry operation, but origOffsets has been cleared.
4.2.Skipping offset commit, no change since last commit
4.3.Finished offset commit successfully in 0 ms for sequence number 384: null
Attachments
Issue Links
- links to