Major Description
This is a followup issue from the discussion in https://github.com/apache/kafka/pull/14836#discussion_r1409739363.
KafkaEventQueue does de-duping and only allows one outstanding CommunicationEvent in the queue. But it seems that duplicated HeartbeatRequests could still be generated. CommunicationEvent calls sendBrokerHeartbeat that calls the following.
_channelManager.sendRequest(new BrokerHeartbeatRequest.Builder(data), handler)
The problem is that we have another queue in NodeToControllerChannelManagerImpl that doesn't do the de-duping. Once a CommunicationEvent is dequeued from KafkaEventQueue, a HeartbeatRequest will be queued in NodeToControllerChannelManagerImpl. At this point, another CommunicationEvent could be enqueued in KafkaEventQueue. When it's processed, another HeartbeatRequest will be queued in NodeToControllerChannelManagerImpl.
This probably won't introduce long lasting duplicated HeartbeatRequest in practice since CommunicationEvent is typically queued in KafkaEventQueue for heartbeat interval. By that time, other pending HeartbeatRequests will be processed and de-duped when enqueuing to KafkaEventQueue. However, duplicated requests could make it hard to reason about tests.
Attachments
Issue Links
- links to