[KAFKA-10119] StreamsResetter fails with TimeoutException for older Brokers

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Duplicate
• Component/s: consumer
• Assignee: Unassigned
• Reporter: Bruno Cadonna
• Created: 08/Jun/20 15:09
• Updated: 11/Jun/20 09:31
• Resolved: 10/Jun/20 19:58

Description

Since somewhere after commit 2d37c8c84 in Apache Kafka, the streams resetter consistently fails with brokers of version confluent-5.0.1.

The following exception is thrown:

org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before the position for partition test-0 could be determined

which comes from this line within the StreamsResetter class:

System.out.println("Topic: " + p.topic() + " Partition: " + p.partition() + " Offset: " + client.position(p));

The exception is not thrown with brokers of version confluent-5.5.0. I have not tried brokers of other versions.

The bug can be reproduced with the following steps:
1. check out commit 7777dc8f8ffd2ad from Apache Kafka
2. build with ./gradlew clean -PscalaVersion=2.13 jar
3. start a confluent-5.0.1 broker.
4. create a topic with bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test
5. start streams resetter with bin/kafka-streams-application-reset.sh --application-id test --bootstrap-servers localhost:9092 --input-topics test

Streams resetter should output:

ERROR: org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before the position for partition test-0 could be determined
org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before the position for partition test-0 could be determined

Issue Links

• is caused by: KAFKA-10123 Regression resetting offsets in consumer when fetching from old broker