RollingFileAppender uses a Mutex to try to prevent multiple processes from rolling the same file. However, I think there is an issue with the implementation.
The Mutex is created with default security. This means that if two processes running under different accounts are configured to log to the same file, one of them will get an UnauthorizedAccessException (caught by log4net) from the Mutex constructor, as the default Mutex security doesn't allow other (non administrator) users to use the Mutex. As a result, one of the processes will be completely unable to log.
The problem can be reproduced by creating two services running under different accounts and configured to log to the same file using RollingFileAppender. If diagnostic logging is enabled, an UnauthorizedAccessException with a stack trace will be logged. If the Mutex were to be created with an access rule that allows everyone to use it, then the exception should disappear. The downside is that others on the system would be able to interfere by grabbing the Mutex, but that is partly possible today if someone creates the Mutex before log4net. Note that trying to reproduce the issue by starting a console application with different accounts might fail, as the applications would share a logon session and be able to use each other's Mutexes. Services run in separate sessions.
There might also be another, lesser issue. The Mutex is created with a name that isn't explicitly in the global namespace. Services use the global namespace by default, but client (user) applications default to an isolated session namespace. This might be by design, but it means that a service and a client application might try to roll the same file simultaneously.
Exception if a process running under a different account has created the Mutex:
log4net:ERROR Could not create Appender [file] of type [log4net.Appender.RollingFileAppender]. Reported error follows.
System.UnauthorizedAccessException: Access to the path 'D__temp_Log4netMutexTest_Log' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.Threading.Mutex.MutexTryCodeHelper.MutexTryCode(Object userData)
at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
at System.Threading.Mutex.CreateMutexWithGuaranteedCleanup(Boolean initiallyOwned, String name, Boolean& createdNew, SECURITY_ATTRIBUTES secAttrs)
at System.Threading.Mutex..ctor(Boolean initiallyOwned, String name, Boolean& createdNew, MutexSecurity mutexSecurity)
at System.Threading.Mutex..ctor(Boolean initiallyOwned, String name)
at log4net.Repository.Hierarchy.XmlHierarchyConfigurator.ParseAppender(XmlElement appenderElement)
Creating a Mutex with an access rule:
var sid = new SecurityIdentifier(WellKnownSidType.WorldSid, null);
var mutexSec = new MutexSecurity();
var mutexRule = new MutexAccessRule(sid, MutexRights.FullControl, AccessControlType.Allow);
var mutex = new Mutex(false, "Log4netMutexServiceTest", out _, mutexSec);