[KAFKA-13770] Regression when Connect uses 0.10.x brokers due to recently added retry logic in KafkaBasedLog

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.5.2, 2.8.2, 3.2.0, 3.1.1, 3.0.2, 2.7.3, 2.6.4
• Fix Version/s: 2.8.2, 3.2.0, 3.1.1, 3.0.2, 2.7.3, 2.6.4
• Component/s: connect
• Assignee: Randall Hauch
• Reporter: Randall Hauch
• Reviewer: Konstantine Karantasis
• Created: 24/Mar/22 20:30
• Updated: 25/Mar/22 04:29
• Resolved: 25/Mar/22 04:29

Description

KAFKA-12879 recently modified Connect's `KafkaBasedLog` class to add retry logic when trying to get the latest offsets for the topic as the `KafkaBasedLog` starts up. This method calls a new method in `TopicAdmin` to read the latest offsets using retries.

When Connect is using an old broker (version 0.10.x or earlier), the old `KafkaBasedLog` logic would catch the `UnsupportedVersionException` thrown by the `TopicAdmin` method, and use the consumer to read offsets instead. The new retry logic unfortunately wrapped the `UnsupportedVersionException` in a `ConnectException`, which means the `KafkaBasedLog` logic doesn't degrade and use the consumer, and instead fails.

The `TopicAdmin.retryEndOffsets(...)` method should propagate the `UnsupportedVersionException` rather than wrapping it. All other exceptions from the admin client are either retriable or already wrapped by a `ConnectException`. Therefore, it appears that `UnsupportedVersionException` is the only special case here.

KAFKA-12879 was backported to a lot of branches (tho only the revert was merged to 2.5), so this new fix should be as well. It does not appear any releases were made from any of those branches with the KAFKA-12879 change.

Issue Links

• is caused by: KAFKA-12879 Compatibility break in Admin.listOffsets()
• links to: GitHub Pull Request #11946