Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Mirror Maker 2 periodically publishes offset sync messages to a Kafka topic that contains the corresponding upstream and downstream offsets for a replicated topic partition.
Currently, this publishing takes place inside the commitRecord method, which is invoked by the Kafka Connect framework after a source record has been successfully sent by its producer (i.e., ack'd by the requested number of brokers).
Mirror Maker 2 also has logic to limit the number of in-flight offset sync messages. Once ten messages have been dispatched to the producer used for offset syncs (which is a separate producer from the one that the Kafka Connect framework uses for sending records received from the poll method) that have not yet been ack'd by the requested number of brokers, Mirror Maker 2 begins to skip sending offset sync messages, and will only resume sending messages once the number of in-flight offset syncs goes below 10, and new calls to the commitRecord method take place.
When bursts of throughput occur in replicated topic partitions, this can cause offset syncs to be dropped for long periods of time if an offset sync is skipped for some topic partition due to a high number of in-flight messages and then no further messages are read from that same topic partition for a while.
Instead, the task should cache offset syncs in its commitRecord method, and only actually send offset sync messages in its commit method, which is invoked periodically by the Kafka Connect framework. Any offset syncs that are skipped due to too many in-flight messages will then be automatically retried later when commit is re-invoked, regardless of whether any more records are read from the corresponding topic partition.
Attachments
Issue Links
- is duplicated by
-
KAFKA-14663 High throughput topics can starve low-throughput MM2 offset syncs
-
- Resolved
-
- links to