[KAFKA-17732] System test coverage for new consumer positions getting behind committed offsets

• Type: Task
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Fix Version/s: 4.1.0
• Component/s: clients, consumer, system tests
• Labels: consumer-threading-refactor
• Assignee: TaiJuWu
• Reporter: Lianet Magrans
• Created: 08/Oct/24 19:19
• Updated: 4 days ago 17:32

Description

We recently catch a bug where the new consumer positions would get behind the committed offsets (fixed with https://issues.apache.org/jira/browse/KAFKA-17674). It was only discovered while adding  a new system test (test_consumer_rolling_migration, not yet in trunk, PR should be available soon in trunk under https://issues.apache.org/jira/browse/KAFKA-17272)

We should check why this failure wasn't catch with existing tests, and add a test for it (test to ensure positions are updated properly, that would fail without KAFKA-17674 and pass with it).