[KAFKA-17493] Sink connector-related OffsetsApiIntegrationTest suite test cases failing more frequently with new consumer/group coordinator

• Type: Test
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: connect, consumer, group-coordinator
• Assignee: David Jacot
• Reporter: Chris Egerton
• Created: 06/Sep/24 14:40
• Updated: 25/Sep/24 15:21

Description

We recently updated trunk to use the new KIP-848 consumer/group coordinator by default, which appears to have led to an uptick in flakiness for the OffsetsApiIntegrationTest suite for Connect (specifically, the test cases that use sink connectors, which makes sense since they're the type of connector that uses a consumer group under the hood).

Gradle Enterprise shows that in the week before that update was made, the test suite had a flakiness rate of about 4% (https://ge.apache.org/scans/tests?search.rootProjectNames=kafka&search.startTimeMax=1724558400000&search.startTimeMin=1723953600000&search.tags=trunk&search.timeZoneId=America%2FNew_York&tests.container=org.apache.kafka.connect.integration.*&tests.sortField=FLAKY), and in the week and a half since, the flakiness rate has jumped to 17% (https://ge.apache.org/scans/tests?search.rootProjectNames=kafka&search.startTimeMax=1725681599999&search.startTimeMin=1724731200000&search.tags=trunk&search.timeZoneId=America%2FNew_York&tests.container=org.apache.kafka.connect.integration.*&tests.sortField=FLAKY).