[KAFKA-14244] Prevent unit/integration tests from terminating JVM

• Type: Improvement
• Status: Patch Available
• Priority: Minor
• Resolution: Unresolved
• Component/s: unit tests
• Assignee: Chris Egerton
• Reporter: Chris Egerton
• Created: 20/Sep/22 14:24
• Updated: 06/Jun/23 17:10

Description

We've seen several cases of unit/integration tests invoking System::exit in the past, which terminates the CI build immediately and blocks us from collecting results for any not-yet-run tests.

In most cases, it's simple enough to use the appropriate wrapper class for calls to System::exit (which is available in both Java and Scala), but if the issue cannot be reproduced locally and instead only occurs during CI builds, it becomes difficult to identify exactly which test is causing the JVM to terminate.

In addition, there are some cases where, even though care is taken to use the correct wrapper class during testing, threads that are leaked during the test end up attempting to terminate the JVM after the wrapper class has been reset, which causes those attempts to actually succeed. See KAFKA-14242 for an example of one such test.

We can explore one or more of these potential improvements:

• Always prevent calls to System::exit from succeeding during tests, both locally and in CI
• Attempt to report which tests are responsible for invoking System::exit directly, so that they can be patched
• Fail the build when System::exit is invoked by a test, in order to surface the issue (which can be either be indicative of poor testing logic, or a genuine bug) during CI builds

Issue Links

• is caused by: KAFKA-10811 System exit from MirrorConnectorsIntegrationTest#testReplication
• is caused by: KAFKA-14242 Hanging logManager in testReloadUpdatedFilesWithoutConfigChange test
• is caused by: KAFKA-14233 Jenkins build failed with task ':core:unitTest' exit value 1
• links to: GitHub Pull Request #12666