[KAFKA-16862] Refactor ConsumerTaskTest to be deterministic and avoid tight loops - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Task
Status: Resolved
Priority: Minor
Resolution: Fixed
Affects Version/s: 3.8.0
Fix Version/s: 3.9.0
Component/s: Tiered-Storage
Labels:
- newbie

Description

The ConsumerTaskTest instantiates a MockConsumer, and uses this MockConsumer instance in the ConsumerTask, which is run in a background thread.

This causes the background thread to tight loop on MockConsumer#poll, which has no sleep or other delay mechanism. This wastes CPU cycles and makes it impossible to use MockConsumer#schedulePollTask to meaningfully mock out the poll behavior.

The test thread then needs to use TestUtils.waitForCondition, which repeatedly polls a result until it is satisfactory, wasting CPU cycles and introducing opportunities for timeout errors. (The test is not currently flaky in CI, so this is less of a concern).

Instead, the ConsumerTaskTest can be rewritten to not utilize a background thread, and make all calls to the MockConsumer on the same thread. This is the model that the DistributedHerderTest uses with DistributedHerder#tick, and WorkerSinkTaskTest uses with WorkerSinkTask#iteration. AbstractWorkerSourceTaskTest uses a similar model with multiple methods, the most notable being AbstractWorkerSourceTask#sendRecords.