[KAFKA-4716] Fix logic for re-checking if internal topic is ready

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.10.2.0
• Fix Version/s: 0.10.2.0, 0.11.0.0
• Component/s: streams
• Labels: architecture
• Assignee: Eno Thereska
• Reporter: Eno Thereska
• Created: 31/Jan/17 13:14
• Updated: 10/Feb/17 07:45
• Resolved: 09/Feb/17 22:01

Description

In InternalTopicManager, we have a hardcoded constant MAX_TOPIC_READY_TRY that is set to 5. We shouldn't hardcode the retry time and it should be based on a timeout, not on a number of retries.

There are cases when the code in makeReady tries to create a topic but then fails because the controller is currently in transition and we get a warning: " Could not create internal topics: Could not create topic: <topic name> due to This is not the correct controller for this cluster." The code proceeds to retry MAX_TOPIC_READY_TRY times in a tight loop, and eventually fails. We should have a retry backoff (perhaps just use retry.backoff.ms) and a timeout (perhaps just use request.timeout.ms) instead of a number of retries.

Issue Links

• links to: GitHub Pull Request #2522
• links to: GitHub Pull Request #2526