[KAFKA-17103] MockClient tight loops when no metadata is present in KafkaProducerTest

• Type: Test
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: clients
• Labels: newbie
• Assignee: zhengke zhou
• Reporter: Greg Harris
• Created: 09/Jul/24 20:33
• Updated: 02/Aug/24 09:02

Description

MockClient can throw this exception:

[2024-07-09 13:19:33,574] ERROR [Producer clientId=producer-1] Uncaught error in kafka producer I/O thread:  (org.apache.kafka.clients.producer.internals.Sender:253)
java.lang.IllegalStateException: No previous metadata update to use
    at org.apache.kafka.clients.MockClient$DefaultMockMetadataUpdater.updateWithCurrentMetadata(MockClient.java:700)
    at org.apache.kafka.clients.MockClient.poll(MockClient.java:323)
    at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:349)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:251)
    at java.base/java.lang.Thread.run(Thread.java:829)

This happens whenever the MockClient.DefaultMockMetadataUpdater#lastUpdate variable is null, which is the case whenever there has not been any mock metadata update.

In practice, this appears to happen in KafkaProducerTest#testTopicExpiryInMetadata, KafkaProducerTest#testTopicRefreshInMetadata, KafkaProducerTest#testTopicNotExistingInMetadata. These three tests also use busy-waiting to have another thread wait for the metadata update request and then provide the metadata update.

We should find some mechanism to allow mocking these metadata updates that avoids this busy waiting/tight looping pattern, as it introduces an opportunity for nondeterminism and wastes CPU cycles.

The solution should remove any usage of Thread.sleep/Utils.sleep/Time.SYSTEM.
Ideally the solution wouldn't require an additional test thread, but this is optional.