[KAFKA-16579] Revert changes to consumer_rolling_upgrade_test.py for the new async Consumer

• Type: Task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.8.0
• Fix Version/s: 3.8.0
• Component/s: clients, consumer, system tests
• Labels: kip-848-client-support system-tests
• Assignee: Philip Nee
• Reporter: Kirk True
• Created: 17/Apr/24 22:31
• Updated: 19/Apr/24 16:00
• Resolved: 19/Apr/24 16:00

Description

To test the new, asynchronous Kafka Consumer implementation, we migrated a slew of system tests to run both the "old" and "new" implementations. KAFKA-16271 updated the system tests in consumer_rolling_upgrade_test.py so it could test the new consumer. However, the test is tailored specifically to the "old" Consumer's protocol and assignment strategy upgrade.

Unsurprisingly, when we run those system tests with the new AsyncKafkaConsumer, we get errors like the following:

test_id:    kafkatest.tests.client.consumer_rolling_upgrade_test.ConsumerRollingUpgradeTest.rolling_update_test.metadata_quorum=ISOLATED_KRAFT.use_new_coordinator=True.group_protocol=consumer
status:     FAIL
run time:   29.634 seconds


    AssertionError("Mismatched assignment: {frozenset(), frozenset({TopicPartition(topic='test_topic', partition=0), TopicPartition(topic='test_topic', partition=1)})}")
Traceback (most recent call last):
  File "/home/jenkins/workspace/system-test-kafka-branch-builder/kafka/venv/lib/python3.7/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/home/jenkins/workspace/system-test-kafka-branch-builder/kafka/venv/lib/python3.7/site-packages/ducktape/tests/runner_client.py", line 262, in run_test
    return self.test_context.function(self.test)
  File "/home/jenkins/workspace/system-test-kafka-branch-builder/kafka/venv/lib/python3.7/site-packages/ducktape/mark/_mark.py", line 433, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/home/jenkins/workspace/system-test-kafka-branch-builder/kafka/tests/kafkatest/tests/client/consumer_rolling_upgrade_test.py", line 77, in rolling_update_test
    self._verify_range_assignment(consumer)
  File "/home/jenkins/workspace/system-test-kafka-branch-builder/kafka/tests/kafkatest/tests/client/consumer_rolling_upgrade_test.py", line 41, in _verify_range_assignment
    "Mismatched assignment: %s" % assignment
AssertionError: Mismatched assignment: {frozenset(), frozenset({TopicPartition(topic='test_topic', partition=0), TopicPartition(topic='test_topic', partition=1)})}

The task here is to revert the changes made in KAFKA-16271.

Issue Links

• relates to: KAFKA-16271 Update consumer_rolling_upgrade_test.py to support KIP-848’s group protocol config
• links to: GitHub Pull Request #15753