[KAFKA-1746] System tests don't handle errors well

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.1.1
• Fix Version/s: 0.9.0.0
• Component/s: system tests
• Assignee: Ewen Cheslack-Postava
• Reporter: Ewen Cheslack-Postava
• Created: 03/Nov/14 19:43
• Updated: 07/Nov/14 02:16
• Resolved: 07/Nov/14 02:16

Description

The system test scripts don't handle errors well. A couple of key issues:

• Unexpected exceptions during tests are just ignored and the tests appear to be successful in the reports.
• The scripts exit code is always 0, even if tests fail.
• Almost no subprocess calls are checked. In a lot of cases this is ok, and sometimes it's not possible (e.g. after starting a long-running remote process), but in some cases such as calls to DumpLogSegments, the tests can miss that the tools is exiting with an exception and the test appears to be successful even though no data was verified.