[KAFKA-15239] producerPerformance system test for old client failed after v3.5.0

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.6.0
• Fix Version/s: 3.6.0
• Component/s: system tests
• Assignee: Federico Valeri
• Reporter: Luke Chen
• Reviewer: Greg Harris
• Created: 24/Jul/23 07:14
• Updated: 10/Aug/23 23:53
• Resolved: 10/Aug/23 21:25

Description

While running producer performance tool in system test for old client (ex: quota_test), we will try to run with the dev-branch's jar file, to make sure it is backward compatible, as described here,.

# In order to ensure more consistent configuration between versions, always use the ProducerPerformance tool from the development branch 

 

But in KAFKA-14525, we're moving tools from core module to a separate tool module, we're actually breaking the backward compatibility. We should fix the system test. Also maybe we should also mention anywhere about this backward compatibility issue?

Note:

This is the command run in system test. Suppose it's testing old client 3.4.0 (file put under `~/Downloads/kafka_2.13-3.4.0` in my env), and running under the latest trunk env.

> for file in ./tools/build/libs/kafka-tools*.jar; do CLASSPATH=$CLASSPATH:$file; done; for file in ./tools/build/dependant-libs*/*.jar; do CLASSPATH=$CLASSPATH:$file; done; export CLASSPATH;  export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:config/tools-log4j.properties"; KAFKA_OPTS= KAFKA_HEAP_OPTS="-XX:+HeapDumpOnOutOfMemoryError" ~/Downloads/kafka_2.13-3.4.0/bin/kafka-run-class.sh org.apache.kafka.tools.ProducerPerformance --topic test_topic --num-records 50000 --record-size 3000 --throughput -1 --producer-props bootstrap.servers=localhost:9092 client.id=overridden_id 


Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/kafka/common/utils/ThroughputThrottler
    at org.apache.kafka.tools.ProducerPerformance.start(ProducerPerformance.java:101)
    at org.apache.kafka.tools.ProducerPerformance.main(ProducerPerformance.java:52)
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.utils.ThroughputThrottler
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
    ... 2 more

 

 

Issue Links

• is related to: KAFKA-14760 Move ThroughputThrottler, break connect-runtime dependency on tools
• links to: GitHub Pull Request #14092
• links to: GitHub Pull Request #14188