[KAFKA-6124] Revisit default config for internal client with regard to resilience

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Abandoned
• Affects Version/s: 1.0.0
• Component/s: streams
• Labels: needs-kip streams-resilience
• Assignee: Unassigned
• Reporter: Matthias J. Sax
• Created: 25/Oct/17 23:25
• Updated: 15/May/20 02:18
• Resolved: 15/May/20 02:18

Description

We should reevaluate the default config of our internally used clients, to update them to make Streams more resilient out-of-the-box.

For example:

• increase producer "retries"
• increase producer "max.block.ms"
• consider impact on max.poll.internal.ms (should we keep it at Integer.MAX_VALUE – note, that KAFKA-5152 resolve the issue why we did set it to infinity)
• double check all other defaults including KafkaAdmintClient

We should also document all finding in the docs and explain how users can configure their application to be more resilient if they want to.

This Jira requires a KIP.

Issue Links

• is related to: KAFKA-6677 Remove EOS producer config max.in.flight.request.per.connection=1
• requires: KAFKA-4857 Replace StreamsKafkaClient with AdminClient in Kafka Streams
• requires: KAFKA-6122 Global Consumer should handle TimeoutException