[KAFKA-12717] Remove internal converter config properties

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: connect
• Labels: needs-kip
• Assignee: Chris Egerton
• Reporter: Chris Egerton
• Reviewer: Randall Hauch
• Created: 26/Apr/21 18:20
• Updated: 02/Jul/21 02:03
• Resolved: 02/Jul/21 02:03

Description

KAFKA-5540 / KIP-174 deprecated but did not officially remove Connect's internal converter worker config properties. With the upcoming 3.0 release, we can make the backwards-incompatible change of completely removing these properties once and for all.

 

One migration path for users who may still be running Connect clusters with different internal converters can be:

1. Stop all workers on the cluster
2. For each internal topic (config, offsets, and status):
   1. Create a new topic to take the place of the existing one
   2. For every message in the existing topic:
      1. Deserialize the message's key and value using the Connect cluster's old internal key and value converters
      2. Serialize the message's key and value using the JSON converter with schemas disabled (by setting the schemas.enable property to false)
      3. Write a message with the new key and value to the new internal topic
3. Reconfigure each Connect worker to use the newly-created internal topics from step 2
4. Start all workers on the cluster

Issue Links

• links to: GitHub Pull Request #10854