Log4j 2
  1. Log4j 2
  2. LOG4J2-164

Add methods is/setEndOfBatch to LogEvent (buffered appender performance improvement)

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0-beta4
    • Fix Version/s: 2.0-beta5
    • Component/s: Core
    • Labels:
      None

      Description

      Many buffered Appenders have an attribute "immediateFlush" that gives users the option to follow each write with a flush to ensure that all messages are available on disk. The drawback of the immediateFlush idiom is the performance impact of touching the disk on every single event.

      If the appender could determine from the current LogEvent whether any more events are expected, it could avoid unnecessary flushes and only write to disk when the last event was received (or the buffer is full).

      This would work especially nicely when receiving bursts of events - a common scenario - when you want to avoid adding more overhead to a system that is already very busy.

      Asynchronous loggers and appenders are well-positioned to use such a mechanism: they can easily call LogEvent.setEndOfBatch(true) if the current event is the last one in the queue.

      The patches I proposed for LOG4J2-151 and LOG4J2-163 already use this mechanism internally. The FastFileAppender included in these patches checks if a log event is a specific class, casts the event to that class, and finally calls the isEndOfBatch method on that event. However I think the idea is generally applicable to any buffered appender and adding these methods to the LogEvent interface would benefit users of the existing AsynchAppender and OutputStreamAppender classes.

      1. LOG4J2-164-patch.txt
        8 kB
        Remko Popma
      2. LOG4J2-164-files.zip
        11 kB
        Remko Popma

        Activity

        Hide
        Remko Popma added a comment -

        Attached LOG4J2-164-files.zip and LOG4J2-164-patch.txt are based on trunk as of Feb 11 (beta4).

        Affected files:
        org/apache/logging/log4j/core/LogEvent
        org/apache/logging/log4j/core/impl/Log4jLogEvent
        org/apache/logging/log4j/core/appender/AsynchAppender
        org/apache/logging/log4j/core/appender/AbstractOutputStreamAppender

        org/apache/logging/log4j/core/lookup/DateLookupTest

        Show
        Remko Popma added a comment - Attached LOG4J2-164 -files.zip and LOG4J2-164 -patch.txt are based on trunk as of Feb 11 (beta4). Affected files: org/apache/logging/log4j/core/LogEvent org/apache/logging/log4j/core/impl/Log4jLogEvent org/apache/logging/log4j/core/appender/AsynchAppender org/apache/logging/log4j/core/appender/AbstractOutputStreamAppender org/apache/logging/log4j/core/lookup/DateLookupTest
        Hide
        Ralph Goers added a comment -

        Patch from LOG4J2-163 applied in revision 1463078. Please verify and close.

        Show
        Ralph Goers added a comment - Patch from LOG4J2-163 applied in revision 1463078. Please verify and close.
        Hide
        Remko Popma added a comment -

        Apologies.
        Looks like my patch for LOG4J2-163 missed the change to AbstractOutputStreamAppender:

        org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender

        current (line 116-117):
        if (this.immediateFlush) {
        manager.flush();

        proposed change:
        if (this.immediateFlush || event.isEndOfBatch()) {
        manager.flush();

        Show
        Remko Popma added a comment - Apologies. Looks like my patch for LOG4J2-163 missed the change to AbstractOutputStreamAppender: org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender current (line 116-117): if (this.immediateFlush) { manager.flush(); proposed change: if (this.immediateFlush || event.isEndOfBatch()) { manager.flush();
        Hide
        Ralph Goers added a comment -

        Change added.

        Show
        Ralph Goers added a comment - Change added.
        Hide
        Remko Popma added a comment -

        Confirmed. Thanks for your prompt response!

        Show
        Remko Popma added a comment - Confirmed. Thanks for your prompt response!

          People

          • Assignee:
            Ralph Goers
            Reporter:
            Remko Popma
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development