[KAFKA-4107] Support offset reset capability in Kafka Connect

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Done
• Fix Version/s: 3.6.0
• Component/s: connect
• Labels: kip
• Assignee: Chris Egerton
• Reporter: Jason Gustafson
• Created: 31/Aug/16 19:27
• Updated: 12/Feb/24 22:20
• Resolved: 23/Jun/23 17:31

Description

It would be useful in some cases to be able to reset connector offsets. For example, if a topic in Kafka corresponding to a source database is accidentally deleted (or deleted because of corrupt data), an administrator may want to reset offsets and reproduce the log from the beginning. It may also be useful to have support for overriding offsets, but that seems like a less likely use case.

Issue Links

• is related to: KAFKA-9070 Kafka Connect - Get the wrong offset value comes from Kafka Connect after increase the number of offset storage topic partition to 3
• is related to: KAFKA-15121 FileStreamSourceConnector and FileStreamSinkConnector should implement KIP-875 APIs (alterOffsets)
• is related to: KAFKA-15177 MirrorMaker 2 should implement the alterOffsets KIP-875 API
• mentioned in: Page Loading...
• mentioned in: Page Loading...

Sub-Tasks

• Implement connector offset write REST API: Resolved
• Implement new STOPPED state for connectors: Resolved
• Implement connector offset reset REST API: Resolved
• Implement connector offset read REST API: Resolved
• Implement connector offset write/reset internal logic: Resolved
• Public documentation for new Kafka Connect offset management REST APIs in 3.5: Resolved
• Public documentation for new Kafka Connect offset management REST APIs: Resolved