[KAFKA-9070] Kafka Connect - Get the wrong offset value comes from Kafka Connect after increase the number of offset storage topic partition to 3

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.3.0
• Component/s: connect
• Labels: connect
• Environment: debezium/connect: 0.10 ; mysql: 5.6 ; kafka: 2.3.0
• Assignee: Unassigned
• Reporter: kaikai.hou
• Created: 21/Oct/19 03:36
• Updated: 12/Feb/24 22:20

Description

I'm using the Debezium project with distributed mode.

Problem:

I found a problem : Get the wrong offset value comes from Kafka Connect after increase the number of offset storage topic partition to 3.

1. Cluster mode, tow node (container);
2. Offset storage topic only have 1 partition;
3. Create 3 mysql connector ;
4. Change data, all connector have offset recored in partition 0.
4. Increase offset storage topic partition number to 3.
5. Change data, and some connector store their offset records to partition 1 or partition 2.
6.* Restart all connect service, then all connector will read offset records from partition 0*.
7. Then, the connector that stored offset records to partition 1 or partition 2 will get repeat data. (the offset records in partition 0 are* too old* for this connectors)

Debug
The Debezium developer checked their code found that the partition handling is delegated to Kafka Connect. their reply

Then, after they reproduced the problem, they found that the problem is that the incorrect offset is returned by Kafka Connect .

Issue Links

• relates to: KAFKA-4107 Support offset reset capability in Kafka Connect