Log4j 2
  1. Log4j 2
  2. LOG4J2-164

Add methods is/setEndOfBatch to LogEvent (buffered appender performance improvement)

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0-beta4
    • Fix Version/s: 2.0-beta5
    • Component/s: Core
    • Labels:
      None

      Description

      Many buffered Appenders have an attribute "immediateFlush" that gives users the option to follow each write with a flush to ensure that all messages are available on disk. The drawback of the immediateFlush idiom is the performance impact of touching the disk on every single event.

      If the appender could determine from the current LogEvent whether any more events are expected, it could avoid unnecessary flushes and only write to disk when the last event was received (or the buffer is full).

      This would work especially nicely when receiving bursts of events - a common scenario - when you want to avoid adding more overhead to a system that is already very busy.

      Asynchronous loggers and appenders are well-positioned to use such a mechanism: they can easily call LogEvent.setEndOfBatch(true) if the current event is the last one in the queue.

      The patches I proposed for LOG4J2-151 and LOG4J2-163 already use this mechanism internally. The FastFileAppender included in these patches checks if a log event is a specific class, casts the event to that class, and finally calls the isEndOfBatch method on that event. However I think the idea is generally applicable to any buffered appender and adding these methods to the LogEvent interface would benefit users of the existing AsynchAppender and OutputStreamAppender classes.

      1. LOG4J2-164-patch.txt
        8 kB
        Remko Popma
      2. LOG4J2-164-files.zip
        11 kB
        Remko Popma

        Activity

        Remko Popma created issue -
        Remko Popma made changes -
        Field Original Value New Value
        Description Many buffered Appenders have an attribute "immediateFlush" that gives users the option to follow each write with a flush to ensure that all messages are available on disk. The drawback of the immediateFlush idiom is the performance impact of touching the disk on every single event.

        If the appender could determine from the current LogEvent whether any more events are expected, it could avoid unnecessary flushes and only write to disk when the last event was received (or the buffer is full).

        This would work especially nicely when receiving bursts of events - a common scenario - when you want to avoid adding more overhead to a system that is already very busy.

        Asynchronous loggers and appenders are well-positioned to use such a mechanism: they can easily call LogEvent.setEndOfBatch(true) if the current event is the last one in the queue.

        The patches I proposed for LOG4J2-151 and LOG4J2-163 already use this mechanism internally with the FastFileAppender included in these patches, but I think the idea is generally applicable to any buffered appender.
        Many buffered Appenders have an attribute "immediateFlush" that gives users the option to follow each write with a flush to ensure that all messages are available on disk. The drawback of the immediateFlush idiom is the performance impact of touching the disk on every single event.

        If the appender could determine from the current LogEvent whether any more events are expected, it could avoid unnecessary flushes and only write to disk when the last event was received (or the buffer is full).

        This would work especially nicely when receiving bursts of events - a common scenario - when you want to avoid adding more overhead to a system that is already very busy.

        Asynchronous loggers and appenders are well-positioned to use such a mechanism: they can easily call LogEvent.setEndOfBatch(true) if the current event is the last one in the queue.

        The patches I proposed for LOG4J2-151 and LOG4J2-163 already use this mechanism internally. The FastFileAppender included in these patches checks if a log event is a specific class, casts the event to that class, and finally calls the isEndOfBatch method on that event. However I think the idea is generally applicable to any buffered appender and adding these methods to the LogEvent interface would benefit users of the existing AsynchAppender and OutputStreamAppender classes.
        Remko Popma made changes -
        Attachment LOG4J2-164-files.zip [ 12568798 ]
        Attachment LOG4J2-164-patch.txt [ 12568799 ]
        Ralph Goers made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Ralph Goers [ ralph.goers@dslextreme.com ]
        Fix Version/s 2.0-beta5 [ 12323970 ]
        Resolution Fixed [ 1 ]
        Remko Popma made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Ralph Goers
            Reporter:
            Remko Popma
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development