[LOG4J2-2628] Out of Memory issue log4j2 Async Appender stops

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 2.11.1
• Fix Version/s: 2.11.1
• Component/s: Core

Environment

Linux 

Log4j 2.11.1

JRE 8

 

 

• Assignee: Unassigned
• Reporter: Sampath
• Created: 07/Jun/19 12:45
• Updated: 07/Jun/19 12:50

Description

In one of the systems observed below out of memory

```java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3332) ~[?:1.8.0_181]
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124) ~[?:1.8.0_181]
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448) ~[?:1.8.0_181]
at java.lang.StringBuilder.append(StringBuilder.java:136) ~[?:1.8.0_181]
at org.apache.logging.log4j.core.pattern.LineSeparatorPatternConverter.format(LineSeparatorPatternConverter.java:66) ~[log4j-core-2.11.1.jar:2.11.1]
at org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:38) ~[log4j-core-2.11.1.jar:2.11.1]
at org.apache.logging.log4j.core.layout.PatternLayout$PatternSerializer.toSerializable(PatternLayout.java:334) ~[log4j-core-2.11.1.jar:2.11.1]
at org.apache.logging.log4j.core.layout.PatternLayout.toText(PatternLayout.java:233) ~[log4j-core-2.11.1.jar:2.11.1]
at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:218) ~[log4j-core-2.11.1.jar:2.11.1]
at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) ~[log4j-core-2.11.1.jar:2.11.1]
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) ~[log4j-core-2.11.1.jar:2.11.1] ```

Here out of memory can be due to application issue, However system didn't recover as the consumer thread AsyncAppender$AsyncThread got stopped

Later all log4j producer thread starts waiting below, This results in not releasing the heap. the only option to recover it by restart the system.  Is there any fix/option available, so that producer thread won't block and Async consumer thread recovers.

```
java.lang.Thread.State: WAITING
at sun.misc.Unsafe.park(Native Method)

• waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1917df7d
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
  at java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:353)
  at org.apache.logging.log4j.core.appender.AsyncAppender.logMessageInBackgroundThread(AsyncAppender.java:204)
  at org.apache.logging.log4j.core.async.EventRoute$1.logMessage(EventRoute.java:51)
  at org.apache.logging.log4j.core.appender.AsyncAppender.append(AsyncAppender.java:170)
  at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156)
  at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129)
  at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120)
  at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
  at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:464)
  at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:448)
  at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:431)
  at org.apache.logging.log4j.core.config.LoggerConfig.logParent(LoggerConfig.java:455)
  at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:450)
  at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:431)
  at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:406)
  at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63)
  at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146)
  at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170)
  at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) ```