[KAFKA-16149] Aggressively expire unused client connections

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: clients, consumer, producer
• Assignee: Kirk True
• Reporter: Kirk True
• Created: 16/Jan/24 23:19
• Updated: 16/Jan/24 23:19

Description

The goal is to minimize the number of connections from the client to the brokers.

On the Java client, there are potentially two types of network connections to brokers:

1. Connections for metadata requests
2. Connections for fetch, produce, etc. requests

The idea is to apply a much shorter idle time to client connections that have only served metadata (type 1 above) so that they become candidates for expiration more quickly.

Alternatively (or additionally), a change to the way metadata requests are routed could be made to reduce the number of connections.