Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
None
-
None
-
None
Description
Although the system test code is also part of the open source repository, not too much attention is paid to this module today. The incurred results is that we keep breaking the system tests with either changes on the admin tools, or library upgrades that change the APIs like Zookeeper. And when the system tests breaks / hangs / etc, it is also hard to debug the issue. We need to treat the system test suite just as part of the open source code.
Based on my personal experience trouble shooting system tests, I would propose doing at least the follow enhancement around system tests.
1. Add unit tests for all system util test tools, for example:
kafka_system_test_utils.get_controller_attributes
kafka_system_test_utils.get_leader_for
2. Add exception handling logic in the python test framework to clean-up the testbed upon failures, so that the subsequent test cases will not be affected.
3. Remove timing based mechanism such as "sleep(5000) to wait for metadata to be propagated" as much as possible to avoid transient failures.
After those enhancements, we should probably also pick a very small subset (say one from each suite) of the system test cases into the patch reviewing process along with the unit tests.
Attachments
1.
|
Improve system test configuration |
|
Closed | Gwen Shapira |
2.
|
Improve system test documentation |
|
Resolved | Ewen Cheslack-Postava |