Stefania discovered that tests that don't produce a lot of log output end up producing 0 debug output to files because the data is not flushed as part of the shutdown hook. I traced through and it looks like the shutdown hook doesn't actually invoke code that does anything useful. It shuts down an executor service in the logging context but doesn't call stop on any appenders.
A hackish thing we can do is use a status listener to collect all the appenders and then stop them when the shutdown hook runs. Even adding a small delay to the shutdown hook (no code changes on our part) would in let the async appender flush in 90% of cases.
We still need to fix it for test which uses a different config file and for which a small delay is not desirable.