[KAFKA-10131] Minimize use of --zookeeper flag in ducktape tests

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.7.0
• Component/s: system tests
• Assignee: Ron Dagostino
• Reporter: Vinoth Chandar
• Reviewer: Colin McCabe
• Created: 09/Jun/20 21:25
• Updated: 15/Sep/20 17:07
• Resolved: 15/Sep/20 17:04

Description

Get the ducktape tests working without the --zookeeper flag (except for scram).

(Note: When doing compat testing we'll still use the old flags.)

Below are the current usages 

[tests]$ grep -R -e "--zookeeper" . 
./kafkatest/tests/core/zookeeper_tls_encrypt_only_test.py:        # Cannot use --zookeeper because kafka-topics.sh is unable to connect to a TLS-enabled ZooKeeper quorum,

./kafkatest/tests/client/quota_test.py:        cmd = "%s --zookeeper %s --alter --add-config producer_byte_rate=%d,consumer_byte_rate=%d" % \
./kafkatest/services/console_consumer.py:            cmd += " --zookeeper %(zk_connect)s" % args

./kafkatest/services/security/security_config.py:            cmd = "%s --zookeeper %s --entity-name %s --entity-type users --alter --add-config %s=[password=%s]" % \

./kafkatest/services/zookeeper.py:        la_migra_cmd += "%s --zookeeper.acl=%s --zookeeper.connect=%s %s" % \
./kafkatest/services/zookeeper.py:        cmd = "%s kafka.admin.ConfigCommand --zookeeper %s %s --describe --topic %s" % \

# Used by MessageFormatChangeTest, TruncationTest 
./kafkatest/services/kafka/kafka.py:        cmd += "%s --zookeeper %s %s --entity-name %s --entity-type topics --alter --add-config message.format.version=%s" % \
./kafkatest/services/kafka/kafka.py:        cmd += "%s --zookeeper %s %s --entity-name %s --entity-type topics --alter --add-config unclean.leader.election.enable=%s" % \


# called by reassign_partitions.sh, ThrottlingTest, ReassignPartitionsTest  
./kafkatest/services/kafka/kafka.py:        cmd += "--zookeeper %s " % self.zk_connect_setting()
./kafkatest/services/kafka/kafka.py:        cmd += "--zookeeper %s " % self.zk_connect_setting()
./kafkatest/services/kafka/kafka.py:            connection_setting = "--zookeeper %s" % (self.zk_connect_setting())
 

Issue Links

• fixes: KAFKA-10213 Prefer --bootstrap-server in ducktape tests for Kafka clients
• fixes: KAFKA-10258 Get rid of use_zk_connection flag in kafka.py public methods
• links to: GitHub Pull Request #9274

Sub-Tasks

• Prefer --bootstrap-server for reassign_partitions command in ducktape tests: Resolved
• Prefer --bootstrap-server ducktape tests using kafka_configs.sh: Resolved
• Prefer --bootstrap-server in ducktape tests for Kafka clients: Resolved
• Get rid of use_zk_connection flag in kafka.py public methods: Resolved