In a production environment, under load the following scenario developed: - Async buffer filled up, and the proceeding requests took priority over the Dispatcher, - additional requests were blocked, - as the system finished enough requests so that the low prio Dispatcher could run, it proceeded with its appender calls - each event processed, freed another slot for one of the blocked threads, which in turn immediately took over the CPU from the Dispatcher. The end effect was a severly choked system. Even worse, on Solaris the blocked threads are not scheduled fairly, but in nearly-LIFO order, leading to near-arbitrary delays. The priority for the Dispatcher should be normal by default. I don't mind if there are niche use cases which require a low prio for the Dispatcher, but that should then be made optionally configurable. I can provide the simple change, plus a fix for the race condition which can develop between the Dispatcher and the blocked threads when the Dispatcher manages to process 2 events before the next thread inserts one. Drop me a note. To test for this scenario, you need a slow Appender such as a DB appender, or add an artificial delay. As far as I know, this applies 1.2.8 (even though I looked at 1.2.4 for the fix) Martin
Ouch. We would appreciate a unified diff against the HEAD of log4j cvs if you can. I agree that a default of normal priority would be nice, however I am wondering whether we should maintain the backward 'compatibility' and leave the default priority as it is now, and we could expose the priority as a Javabean property to be modified via the config. We would then need to document clearly the limitations of the priority under stress. If you have any time to create a junit test to reproduce the current problem prior to the change and verify that the the change fixes the issue, that would be even better! :) cheers, Paul Smith
Marking this as a duplicate of 38137 which I'm using as a catch all for blocking issues with the AsyncAppender. A new AsyncAppender implementation was added for bug 38982 which hopefully will also eliminate the deadlock issues observed. The Dispatcher thread does not have its priority assigned, though that could be added as a configuration parameter. If you a test platform and could test the new implementation (the first attachment describes how to adapt it to log4j 1.2), it would be appreciated. Sorry that it took so long. *** This bug has been marked as a duplicate of 38137 ***