[KAFKA-16776] Builds flaky-failing with leaked client-metrics-reaper thread

• Type: Test
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.8.0
• Labels: flaky-test
• Assignee: Unassigned
• Reporter: Greg Harris
• Created: 15/May/24 19:14
• Updated: 15/May/24 19:17

Description

I observed the assertion added by KAFKA-16477 failing in my build:

https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-15469/6/tests Reproduced here:

Build / JDK 17 and Scala 2.13 / initializationError – org.apache.kafka.tools.TopicCommandIntegrationTest<1s
Build / JDK 17 and Scala 2.13 / initializationError – org.apache.kafka.tools.consumer.group.AuthorizerIntegrationTest<1s
Build / JDK 17 and Scala 2.13 / initializationError – org.apache.kafka.tools.consumer.group.DescribeConsumerGroupTest<1s
Build / JDK 17 and Scala 2.13 / initializationError – org.apache.kafka.tools.consumer.group.ResetConsumerGroupOffsetTest<1s
Build / JDK 17 and Scala 2.13 / initializationError – org.apache.kafka.tools.reassign.ReassignPartitionsIntegrationTest<1s

The exceptions for these tests are as follows:

org.opentest4j.AssertionFailedError: Found 1 unexpected threads during @BeforeAll: `executor-client-metrics` ==> expected: <true> but was: <false>
    at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
    at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
    at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
    at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
    at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:214)
    at app//kafka.utils.TestUtils$.verifyNoUnexpectedThreads(TestUtils.scala:1896)
    at app//kafka.server.QuorumTestHarness.setUpClass(QuorumTestHarness.scala:474)
...
    Suppressed: org.opentest4j.AssertionFailedError: Found 1 unexpected threads during @AfterAll: `executor-client-metrics` ==> expected: <true> but was: <false>
        at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
        at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
        at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
        at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
        at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:214)
        at app//kafka.utils.TestUtils$.verifyNoUnexpectedThreads(TestUtils.scala:1896)
        at app//kafka.server.QuorumTestHarness.tearDownClass(QuorumTestHarness.scala:482)

But there is not any additional context about who leaked this thread.

Issue Links

• is related to: KAFKA-16072 Create Junit 5 extension to detect thread leak
• is related to: KAFKA-16477 Detect thread leaked client-metrics-reaper in tests