Uploaded image for project: 'Log4j 2'
  1. Log4j 2
  2. LOG4J2-2628

Out of Memory issue log4j2 Async Appender stops

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 2.11.1
    • Fix Version/s: 2.11.1
    • Component/s: Core
    • Labels:
      None
    • Environment:

      Linux 

      Log4j 2.11.1

      JRE 8

       

       

      Description

      In one of the systems observed below out of memory

      ```java.lang.OutOfMemoryError: Java heap space
      at java.util.Arrays.copyOf(Arrays.java:3332) ~[?:1.8.0_181]
      at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124) ~[?:1.8.0_181]
      at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448) ~[?:1.8.0_181]
      at java.lang.StringBuilder.append(StringBuilder.java:136) ~[?:1.8.0_181]
      at org.apache.logging.log4j.core.pattern.LineSeparatorPatternConverter.format(LineSeparatorPatternConverter.java:66) ~[log4j-core-2.11.1.jar:2.11.1]
      at org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:38) ~[log4j-core-2.11.1.jar:2.11.1]
      at org.apache.logging.log4j.core.layout.PatternLayout$PatternSerializer.toSerializable(PatternLayout.java:334) ~[log4j-core-2.11.1.jar:2.11.1]
      at org.apache.logging.log4j.core.layout.PatternLayout.toText(PatternLayout.java:233) ~[log4j-core-2.11.1.jar:2.11.1]
      at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:218) ~[log4j-core-2.11.1.jar:2.11.1]
      at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58) ~[log4j-core-2.11.1.jar:2.11.1]
      at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) ~[log4j-core-2.11.1.jar:2.11.1] ```

      Here out of memory can be due to application issue, However system didn't recover as the consumer thread AsyncAppender$AsyncThread got stopped

      Later all log4j producer thread starts waiting below, This results in not releasing the heap. the only option to recover it by restart the system.  Is there any fix/option available, so that producer thread won't block and Async consumer thread recovers.

      ```
      java.lang.Thread.State: WAITING
      at sun.misc.Unsafe.park(Native Method)

      • waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1917df7d
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:353)
        at org.apache.logging.log4j.core.appender.AsyncAppender.logMessageInBackgroundThread(AsyncAppender.java:204)
        at org.apache.logging.log4j.core.async.EventRoute$1.logMessage(EventRoute.java:51)
        at org.apache.logging.log4j.core.appender.AsyncAppender.append(AsyncAppender.java:170)
        at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156)
        at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129)
        at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120)
        at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
        at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:464)
        at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:448)
        at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:431)
        at org.apache.logging.log4j.core.config.LoggerConfig.logParent(LoggerConfig.java:455)
        at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:450)
        at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:431)
        at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:406)
        at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63)
        at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146)
        at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170)
        at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125) ```

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ksampath Sampath
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: