[KAFKA-10021] When reading to the end of the config log, check if fetch.max.wait.ms is greater than worker.sync.timeout.ms

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0, 2.4.0, 2.5.0, 2.6.0, 2.7.0
• Fix Version/s: 2.5.2, 2.8.0, 2.7.1, 2.6.2
• Component/s: connect
• Assignee: Randall Hauch
• Reporter: Sanjana Kaundinya
• Created: 19/May/20 08:18
• Updated: 22/Feb/21 16:11
• Resolved: 10/Feb/21 01:52

Description

Currently in the Connect code in DistributedHerder.java, we see the following piece of code

 

{{ if (!canReadConfigs && !readConfigToEnd(workerSyncTimeoutMs))
return; // Safe to return and tick immediately because readConfigToEnd will do the backoff for us}}

where the workerSyncTimeoutMs passed in is the timeout given to read to the end of the config log. This is a bug as we should check if fetch.wait.max.ms is greater than worker.sync.timeout.ms and if it is, use worker.sync.timeout.ms as the fetch.wait.max.ms. A better fix would be to use the AdminClient to read to the end of the log, but at a minimum we should check the configs.

Issue Links

• causes: KAFKA-12326 MM2 fails to start to due missing required configuration "bootstrap.servers"
• causes: KAFKA-12339 Add retry to admin client's listOffsets
• causes: KAFKA-12340 Recent change to use SharedTopicAdmin results in potential resource leak in deprecated backing store constructors
• links to: GitHub Pull Request #9780