[KAFKA-14569] Migrate EmbeddedKafkaCluster used by Connect integration tests from EmbeddedZookeeper to KRaft - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Task
Status: Resolved
Priority: Blocker
Resolution: Done
Affects Version/s: None
Fix Version/s: 3.9.0
Component/s: connect
Labels:
None

Description

ZooKeeper mode is going to be deprecated in Apache Kafka 4.0. Connect currently uses an EmbeddedKafkaCluster (that depends on an EmbeddedZookeeper) for integration tests with the EmbeddedConnectCluster. This should be migrated to remove the ZooKeeper dependency, instead working in the KRaft mode (probably with co-located brokers and controllers). We could potentially leverage the existing test kit for KRaft clusters which handles a bunch of stuff including the listeners configuration setup, formatting the metadata log directory, allowing usage of non-static random ports for `controller.quorum.voters`, initialization of the shared server(s), broker(s), and controller(s) etc.

One more thing to note is that some Connect integration tests currently use the `kafka.security.authorizer.AclAuthorizer` which requires ZooKeeper. These tests should be migrated to use the new authorizer from KIP-801 if we want to completely eliminate the dependency on ZooKeeper.