This test case has a bit of a race, since it assumes that the logs will be truncated immediately upon job completion.
This isn't the case, since the logs aren't added to the truncation manager until the JVM finished, which can be several seconds after the last task finishes (eg when sleepTimeBeforeSigKill is 5 seconds). So, I found that this test was flaky as is.
You can show this by adding a 10 second sleep before truncating the logs, for example.
This delta patch has the tests loop for 20 seconds while checking the logs for truncation. It only fails if the logs aren't truncated after 20 seconds.