[KAFKA-17661] Fix flaky BufferPoolTest.testBlockTimeout - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: In Progress
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: clients, unit tests
Labels:
None

Description

4 flaky out of 221 trunk build in the past 28 days. (github) (Report Link)

(Sep 27 2024 at 04:54:00)

org.opentest4j.AssertionFailedError: The buffer allocated more memory than its maximum value 10	

at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38)	
at org.junit.jupiter.api.Assertions.fail(Assertions.java:138)	
at org.apache.kafka.clients.producer.internals.BufferPoolTest.testBlockTimeout(BufferPoolTest.java:184)	
at java.lang.reflect.Method.invoke(Method.java:566)	
at java.util.ArrayList.forEach(ArrayList.java:1541)	
at java.util.ArrayList.forEach(ArrayList.java:1541)

Root cause:

The test relies on 3 asynchronous threads being triggered in parallel with the test thread [1]. However, there is no guarantee of parallelism in test environment. The issue will happend if test thread didn't get CPU within 25 ms. We could reproduce this issue by adding 30 ms delay to test thread. Please check the attached patch.
Since a 25 ms delay is obviously unreliable in the test environment, we could consider rewriting the test or increasing the delay. (The maxBlockTimeMs was reduced from 2000ms to 10 ms in ~~KAFKA-9852~~)

[1] https://github.com/apache/kafka/blob/40360819bb97d6b05dfef6451888b4d908fc3bf4/clients/src/test/java/org/apache/kafka/clients/producer/internals/BufferPoolTest.java#L175-L179