Index: pom.xml =================================================================== --- pom.xml (revision 1467676) +++ pom.xml (working copy) @@ -62,7 +62,7 @@ Remko Popma remkop@yahoo.com - log4j-async module development, testing and documentation + Async Low-Latency Loggers, log4j JMX support +9 @@ -116,6 +116,7 @@ 2.0-beta5 seventh 1.9.2 + 3.0.0 org.apache.logging.${project.artifactId} org.apache.logging.log4j.*;version=${project.version};-noimport:=true @@ -261,6 +262,11 @@ provided + com.lmax + disruptor + ${disruptor.version} + + junit junit 4.7 @@ -386,6 +392,8 @@ + + en @@ -569,7 +577,6 @@ web taglib samples - log4j-async Index: core/pom.xml =================================================================== --- core/pom.xml (revision 1467676) +++ core/pom.xml (working copy) @@ -44,6 +44,11 @@ provided + com.lmax + disruptor + true + + org.codehaus.jackson jackson-core-asl true Index: core/src/main/java/org/apache/logging/log4j/core/appender/FastFileAppender.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/appender/FastFileAppender.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/appender/FastFileAppender.java (working copy) @@ -0,0 +1,162 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.appender; + +import java.util.HashMap; +import java.util.Map; + +import org.apache.logging.log4j.core.Filter; +import org.apache.logging.log4j.core.Layout; +import org.apache.logging.log4j.core.LogEvent; +import org.apache.logging.log4j.core.config.Configuration; +import org.apache.logging.log4j.core.config.plugins.Plugin; +import org.apache.logging.log4j.core.config.plugins.PluginAttr; +import org.apache.logging.log4j.core.config.plugins.PluginConfiguration; +import org.apache.logging.log4j.core.config.plugins.PluginElement; +import org.apache.logging.log4j.core.config.plugins.PluginFactory; +import org.apache.logging.log4j.core.layout.PatternLayout; +import org.apache.logging.log4j.core.net.Advertiser; + +/** + * File Appender. + */ +@Plugin(name = "FastFile", type = "Core", elementType = "appender", printObject = true) +public final class FastFileAppender extends AbstractOutputStreamAppender { + + private final String fileName; + private Object advertisement; + private final Advertiser advertiser; + + private FastFileAppender(String name, Layout layout, Filter filter, + FastFileManager manager, String filename, boolean handleException, + boolean immediateFlush, Advertiser advertiser) { + super(name, layout, filter, handleException, immediateFlush, manager); + if (advertiser != null) { + Map configuration = new HashMap( + layout.getContentFormat()); + configuration.putAll(manager.getContentFormat()); + configuration.put("contentType", layout.getContentType()); + configuration.put("name", name); + advertisement = advertiser.advertise(configuration); + } + this.fileName = filename; + this.advertiser = advertiser; + } + + @Override + public void stop() { + super.stop(); + if (advertiser != null) { + advertiser.unadvertise(advertisement); + } + } + + /** + * Write the log entry rolling over the file when required. + * + * @param event The LogEvent. + */ + @Override + public void append(LogEvent event) { + + // Leverage the nice batching behaviour of async Loggers/Appenders: + // we can signal the file manager that it needs to flush the buffer + // to disk at the end of a batch. + // From a user's point of view, this means that all log events are + // _always_ available in the log file, without incurring the overhead + // of immediateFlush=true. + ((FastFileManager) getManager()).setEndOfBatch(event.isEndOfBatch()); + super.append(event); + } + + /** + * Returns the file name this appender is associated with. + * + * @return The File name. + */ + public String getFileName() { + return this.fileName; + } + + // difference from standard File Appender: + // locking is not supported and buffering cannot be switched off + /** + * Create a File Appender. + * + * @param fileName The name and path of the file. + * @param append "True" if the file should be appended to, "false" if it + * should be overwritten. The default is "true". + * @param name The name of the Appender. + * @param immediateFlush "true" if the contents should be flushed on every + * write, "false" otherwise. The default is "true". + * @param suppress "true" if exceptions should be hidden from the + * application, "false" otherwise. The default is "true". + * @param layout The layout to use to format the event. If no layout is + * provided the default PatternLayout will be used. + * @param filter The filter, if any, to use. + * @param advertise "true" if the appender configuration should be + * advertised, "false" otherwise. + * @param advertiseURI The advertised URI which can be used to retrieve the + * file contents. + * @param config The Configuration. + * @return The FileAppender. + */ + @PluginFactory + public static FastFileAppender createAppender( + @PluginAttr("fileName") String fileName, + @PluginAttr("append") String append, + @PluginAttr("name") String name, + @PluginAttr("immediateFlush") String immediateFlush, + @PluginAttr("suppressExceptions") String suppress, + @PluginElement("layout") Layout layout, + @PluginElement("filters") final Filter filter, + @PluginAttr("advertise") final String advertise, + @PluginAttr("advertiseURI") final String advertiseURI, + @PluginConfiguration final Configuration config) { + + boolean isAppend = append == null ? true : Boolean.valueOf(append); + boolean isFlush = immediateFlush == null ? true : Boolean + .valueOf(immediateFlush); + boolean handleExceptions = suppress == null ? true : Boolean + .valueOf(suppress); + boolean isAdvertise = advertise == null ? false : Boolean + .valueOf(advertise); + + if (name == null) { + LOGGER.error("No name provided for FileAppender"); + return null; + } + + if (fileName == null) { + LOGGER.error("No filename provided for FileAppender with name " + + name); + return null; + } + + FastFileManager manager = FastFileManager.getFileManager(fileName, + isAppend, isFlush, advertiseURI); + if (manager == null) { + return null; + } + if (layout == null) { + layout = PatternLayout.createLayout(null, null, null, null); + } + return new FastFileAppender(name, layout, filter, manager, fileName, + handleExceptions, isFlush, isAdvertise ? config.getAdvertiser() + : null); + } +} Index: core/src/main/java/org/apache/logging/log4j/core/appender/FastFileManager.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/appender/FastFileManager.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/appender/FastFileManager.java (working copy) @@ -0,0 +1,207 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.appender; + +import java.io.File; +import java.io.IOException; +import java.io.OutputStream; +import java.io.RandomAccessFile; +import java.nio.ByteBuffer; +import java.util.HashMap; +import java.util.Map; + +/** + * Extends OutputStreamManager but instead of using a buffered output stream, + * this class uses a {@code ByteBuffer} and a {@code RandomAccessFile} to do the + * I/O. + */ +public class FastFileManager extends OutputStreamManager { + private static final int DEFAULT_BUFFER_SIZE = 256 * 1024; + + private static final FastFileManagerFactory FACTORY = new FastFileManagerFactory(); + + private final boolean isImmediateFlush; + private final String advertiseURI; + private final RandomAccessFile randomAccessFile; + private final ByteBuffer buffer; + private ThreadLocal isEndOfBatch = new ThreadLocal(); + + protected FastFileManager(RandomAccessFile file, String fileName, + OutputStream os, boolean immediateFlush, String advertiseURI) { + super(os, fileName); + this.isImmediateFlush = immediateFlush; + this.randomAccessFile = file; + this.advertiseURI = advertiseURI; + isEndOfBatch.set(Boolean.FALSE); + + // TODO make buffer size configurable? + buffer = ByteBuffer.allocate(DEFAULT_BUFFER_SIZE); + } + + /** + * Returns the FastFileManager. + * + * @param fileName The name of the file to manage. + * @param append true if the file should be appended to, false if it should + * be overwritten. + * @param isFlush true if the contents should be flushed to disk on every + * write + * @param advertiseURI the URI to use when advertising the file + * @return A FastFileManager for the File. + */ + public static FastFileManager getFileManager(String fileName, + boolean append, boolean isFlush, String advertiseURI) { + return (FastFileManager) getManager(fileName, new FactoryData(append, + isFlush, advertiseURI), FACTORY); + } + + public Boolean isEndOfBatch() { + return isEndOfBatch.get(); + } + + public void setEndOfBatch(boolean isEndOfBatch) { + this.isEndOfBatch.set(Boolean.valueOf(isEndOfBatch)); + } + + @Override + protected synchronized void write(byte[] bytes, int offset, int length) { + super.write(bytes, offset, length); // writes to dummy output stream + + if (length > buffer.remaining()) { + flush(); + } + buffer.put(bytes, offset, length); + if (isImmediateFlush || isEndOfBatch.get() == Boolean.TRUE) { + flush(); + } + } + + @Override + public void flush() { + buffer.flip(); + try { + randomAccessFile.write(buffer.array(), 0, buffer.limit()); + } catch (IOException ex) { + String msg = "Error writing to RandomAccessFile " + getName(); + throw new AppenderRuntimeException(msg, ex); + } + buffer.clear(); + } + + @Override + public void close() { + flush(); + try { + randomAccessFile.close(); + } catch (final IOException ex) { + LOGGER.error("Unable to close RandomAccessFile " + getName() + ". " + + ex); + } + } + + /** + * Returns the name of the File being managed. + * + * @return The name of the File being managed. + */ + public String getFileName() { + return getName(); + } + + /** {@code OutputStream} subclass that does not write anything. */ + private static class DummyOutputStream extends OutputStream { + @Override + public void write(int b) throws IOException { + } + + @Override + public void write(byte[] b, int off, int len) throws IOException { + } + } + + /** + * FileManager's content format is specified by: + *

+ * Key: "fileURI" Value: provided "advertiseURI" param. + * + * @return Map of content format keys supporting FileManager + */ + public Map getContentFormat() { + Map result = new HashMap( + super.getContentFormat()); + result.put("fileURI", advertiseURI); + return result; + } + + /** + * Factory Data. + */ + private static class FactoryData { + private final boolean append; + private final boolean immediateFlush; + private final String advertiseURI; + + /** + * Constructor. + * + * @param append Append status. + */ + public FactoryData(boolean append, boolean immediateFlush, + String advertiseURI) { + this.append = append; + this.immediateFlush = immediateFlush; + this.advertiseURI = advertiseURI; + } + } + + /** + * Factory to create a FastFileManager. + */ + private static class FastFileManagerFactory implements + ManagerFactory { + + /** + * Create a FastFileManager. + * + * @param name The name of the File. + * @param data The FactoryData + * @return The FastFileManager for the File. + */ + public FastFileManager createManager(String name, FactoryData data) { + File file = new File(name); + final File parent = file.getParentFile(); + if (null != parent && !parent.exists()) { + parent.mkdirs(); + } + if (!data.append) { + file.delete(); + } + + OutputStream os = new DummyOutputStream(); + RandomAccessFile raf; + try { + raf = new RandomAccessFile(name, "rw"); + return new FastFileManager(raf, name, os, data.immediateFlush, + data.advertiseURI); + } catch (Exception ex) { + LOGGER.error("FastFileManager (" + name + ") " + ex); + } + return null; + } + } + +} Index: core/src/main/java/org/apache/logging/log4j/core/appender/FastRollingFileAppender.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/appender/FastRollingFileAppender.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/appender/FastRollingFileAppender.java (working copy) @@ -0,0 +1,207 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.appender; + +import java.util.HashMap; +import java.util.Map; + +import org.apache.logging.log4j.core.Filter; +import org.apache.logging.log4j.core.Layout; +import org.apache.logging.log4j.core.LogEvent; +import org.apache.logging.log4j.core.appender.rolling.DefaultRolloverStrategy; +import org.apache.logging.log4j.core.appender.rolling.FastRollingFileManager; +import org.apache.logging.log4j.core.appender.rolling.RollingFileManager; +import org.apache.logging.log4j.core.appender.rolling.RolloverStrategy; +import org.apache.logging.log4j.core.appender.rolling.TriggeringPolicy; +import org.apache.logging.log4j.core.config.Configuration; +import org.apache.logging.log4j.core.config.plugins.Plugin; +import org.apache.logging.log4j.core.config.plugins.PluginAttr; +import org.apache.logging.log4j.core.config.plugins.PluginConfiguration; +import org.apache.logging.log4j.core.config.plugins.PluginElement; +import org.apache.logging.log4j.core.config.plugins.PluginFactory; +import org.apache.logging.log4j.core.layout.PatternLayout; +import org.apache.logging.log4j.core.net.Advertiser; + +/** + * An appender that writes to random access files and can roll over at + * intervals. + */ +@Plugin(name = "FastRollingFile", type = "Core", elementType = "appender", printObject = true) +public final class FastRollingFileAppender extends AbstractOutputStreamAppender { + + private final String fileName; + private final String filePattern; + private Object advertisement; + private final Advertiser advertiser; + + private FastRollingFileAppender(String name, Layout layout, + Filter filter, RollingFileManager manager, String fileName, + String filePattern, boolean handleException, + boolean immediateFlush, Advertiser advertiser) { + super(name, layout, filter, handleException, immediateFlush, manager); + if (advertiser != null) { + Map configuration = new HashMap( + layout.getContentFormat()); + configuration.put("contentType", layout.getContentType()); + configuration.put("name", name); + advertisement = advertiser.advertise(configuration); + } + this.fileName = fileName; + this.filePattern = filePattern; + this.advertiser = advertiser; + } + + @Override + public void stop() { + super.stop(); + if (advertiser != null) { + advertiser.unadvertise(advertisement); + } + } + + /** + * Write the log entry rolling over the file when required. + * + * @param event The LogEvent. + */ + @Override + public void append(final LogEvent event) { + FastRollingFileManager manager = (FastRollingFileManager) getManager(); + manager.checkRollover(event); + + // Leverage the nice batching behaviour of async Loggers/Appenders: + // we can signal the file manager that it needs to flush the buffer + // to disk at the end of a batch. + // From a user's point of view, this means that all log events are + // _always_ available in the log file, without incurring the overhead + // of immediateFlush=true. + manager.setEndOfBatch(event.isEndOfBatch()); + super.append(event); + } + + /** + * Returns the File name for the Appender. + * + * @return The file name. + */ + public String getFileName() { + return fileName; + } + + /** + * Returns the file pattern used when rolling over. + * + * @return The file pattern. + */ + public String getFilePattern() { + return filePattern; + } + + /** + * Create a FastRollingFileAppender. + * + * @param fileName The name of the file that is actively written to. + * (required). + * @param filePattern The pattern of the file name to use on rollover. + * (required). + * @param append If true, events are appended to the file. If false, the + * file is overwritten when opened. Defaults to "true" + * @param name The name of the Appender (required). + * @param immediateFlush When true, events are immediately flushed. Defaults + * to "true". + * @param policy The triggering policy. (required). + * @param strategy The rollover strategy. Defaults to + * DefaultRolloverStrategy. + * @param layout The layout to use (defaults to the default PatternLayout). + * @param filter The Filter or null. + * @param suppress "true" if exceptions should be hidden from the + * application, "false" otherwise. The default is "true". + * @param advertise "true" if the appender configuration should be + * advertised, "false" otherwise. + * @param advertiseURI The advertised URI which can be used to retrieve the + * file contents. + * @param config The Configuration. + * @return A FastRollingFileAppender. + */ + @PluginFactory + public static FastRollingFileAppender createAppender( + @PluginAttr("fileName") final String fileName, + @PluginAttr("filePattern") final String filePattern, + @PluginAttr("append") final String append, + @PluginAttr("name") final String name, + @PluginAttr("immediateFlush") final String immediateFlush, + @PluginElement("policy") final TriggeringPolicy policy, + @PluginElement("strategy") RolloverStrategy strategy, + @PluginElement("layout") Layout layout, + @PluginElement("filter") final Filter filter, + @PluginAttr("suppressExceptions") final String suppress, + @PluginAttr("advertise") final String advertise, + @PluginAttr("advertiseURI") final String advertiseURI, + @PluginConfiguration final Configuration config) { + + final boolean isAppend = append == null ? true : Boolean + .valueOf(append); + final boolean handleExceptions = suppress == null ? true : Boolean + .valueOf(suppress); + final boolean isFlush = immediateFlush == null ? true : Boolean + .valueOf(immediateFlush); + boolean isAdvertise = advertise == null ? false : Boolean + .valueOf(advertise); + + if (name == null) { + LOGGER.error("No name provided for FileAppender"); + return null; + } + + if (fileName == null) { + LOGGER.error("No filename was provided for FileAppender with name " + + name); + return null; + } + + if (filePattern == null) { + LOGGER.error("No filename pattern provided for FileAppender with name " + + name); + return null; + } + + if (policy == null) { + LOGGER.error("A TriggeringPolicy must be provided"); + return null; + } + + if (strategy == null) { + strategy = DefaultRolloverStrategy.createStrategy(null, null, + "true", config); + } + + final FastRollingFileManager manager = FastRollingFileManager + .getFastRollingFileManager(fileName, filePattern, isAppend, + isFlush, policy, strategy, advertiseURI); + if (manager == null) { + return null; + } + + if (layout == null) { + layout = PatternLayout.createLayout(null, null, null, null); + } + + return new FastRollingFileAppender(name, layout, filter, manager, + fileName, filePattern, handleExceptions, isFlush, + isAdvertise ? config.getAdvertiser() : null); + } +} Index: core/src/main/java/org/apache/logging/log4j/core/appender/rolling/FastRollingFileManager.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/appender/rolling/FastRollingFileManager.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/appender/rolling/FastRollingFileManager.java (working copy) @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.appender.rolling; + +import java.io.File; +import java.io.FileNotFoundException; +import java.io.IOException; +import java.io.OutputStream; +import java.io.RandomAccessFile; +import java.nio.ByteBuffer; + +import org.apache.logging.log4j.core.appender.AppenderRuntimeException; +import org.apache.logging.log4j.core.appender.ManagerFactory; + +/** + * Extends RollingFileManager but instead of using a buffered output stream, + * this class uses a {@code ByteBuffer} and a {@code RandomAccessFile} to do the + * I/O. + */ +public class FastRollingFileManager extends RollingFileManager { + private static final int DEFAULT_BUFFER_SIZE = 256 * 1024; + + private static final FastRollingFileManagerFactory FACTORY = new FastRollingFileManagerFactory(); + + private final boolean isImmediateFlush; + private RandomAccessFile randomAccessFile; + private final ByteBuffer buffer; + private ThreadLocal isEndOfBatch = new ThreadLocal(); + + public FastRollingFileManager(RandomAccessFile raf, String fileName, + String pattern, OutputStream os, boolean append, + boolean immediateFlush, long size, long time, + TriggeringPolicy policy, RolloverStrategy strategy, + String advertiseURI) { + super(fileName, pattern, os, append, size, time, policy, strategy, + advertiseURI); + this.isImmediateFlush = immediateFlush; + this.randomAccessFile = raf; + isEndOfBatch.set(Boolean.FALSE); + + // TODO make buffer size configurable? + buffer = ByteBuffer.allocate(DEFAULT_BUFFER_SIZE); + } + + public static FastRollingFileManager getFastRollingFileManager( + String fileName, String filePattern, boolean isAppend, + boolean immediateFlush, TriggeringPolicy policy, + RolloverStrategy strategy, String advertiseURI) { + return (FastRollingFileManager) getManager(fileName, new FactoryData( + filePattern, isAppend, immediateFlush, policy, strategy, + advertiseURI), FACTORY); + } + + public Boolean isEndOfBatch() { + return isEndOfBatch.get(); + } + + public void setEndOfBatch(boolean isEndOfBatch) { + this.isEndOfBatch.set(Boolean.valueOf(isEndOfBatch)); + } + + @Override + protected synchronized void write(byte[] bytes, int offset, int length) { + super.write(bytes, offset, length); // writes to dummy output stream + + if (length > buffer.remaining()) { + flush(); + } + buffer.put(bytes, offset, length); + if (isImmediateFlush || isEndOfBatch.get() == Boolean.TRUE) { + flush(); + } + } + + @Override + protected void createFileAfterRollover() throws IOException { + this.randomAccessFile = new RandomAccessFile(getFileName(), "rw"); + if (isAppend()) { + randomAccessFile.seek(randomAccessFile.length()); + } + } + + @Override + public void flush() { + buffer.flip(); + try { + randomAccessFile.write(buffer.array(), 0, buffer.limit()); + } catch (IOException ex) { + String msg = "Error writing to RandomAccessFile " + getName(); + throw new AppenderRuntimeException(msg, ex); + } + buffer.clear(); + } + + @Override + public void close() { + flush(); + try { + randomAccessFile.close(); + } catch (final IOException ex) { + LOGGER.error("Unable to close RandomAccessFile " + getName() + ". " + + ex); + } + } + + /** + * Factory to create a FastRollingFileManager. + */ + private static class FastRollingFileManagerFactory implements + ManagerFactory { + + /** + * Create the FastRollingFileManager. + * + * @param name The name of the entity to manage. + * @param data The data required to create the entity. + * @return a RollingFileManager. + */ + public FastRollingFileManager createManager(String name, + FactoryData data) { + File file = new File(name); + final File parent = file.getParentFile(); + if (null != parent && !parent.exists()) { + parent.mkdirs(); + } + if (!data.append) { + file.delete(); + } + long size = data.append ? file.length() : 0; + long time = file.lastModified(); + + RandomAccessFile raf; + try { + raf = new RandomAccessFile(name, "rw"); + return new FastRollingFileManager(raf, name, data.pattern, + new DummyOutputStream(), data.append, + data.immediateFlush, size, time, data.policy, + data.strategy, data.advertiseURI); + } catch (FileNotFoundException ex) { + LOGGER.error("FastRollingFileManager (" + name + ") " + ex); + } + return null; + } + } + + /** {@code OutputStream} subclass that does not write anything. */ + private static class DummyOutputStream extends OutputStream { + @Override + public void write(int b) throws IOException { + } + + @Override + public void write(byte[] b, int off, int len) throws IOException { + } + } + + /** + * Factory data. + */ + private static class FactoryData { + private final String pattern; + private final boolean append; + private final boolean immediateFlush; + private final TriggeringPolicy policy; + private final RolloverStrategy strategy; + private final String advertiseURI; + + /** + * Create the data for the factory. + * + * @param pattern The pattern. + * @param append The append flag. + * @param immediateFlush + */ + public FactoryData(String pattern, boolean append, + boolean immediateFlush, TriggeringPolicy policy, + RolloverStrategy strategy, String advertiseURI) { + this.pattern = pattern; + this.append = append; + this.immediateFlush = immediateFlush; + this.policy = policy; + this.strategy = strategy; + this.advertiseURI = advertiseURI; + } + } + +} Index: core/src/main/java/org/apache/logging/log4j/core/async/AsyncLogger.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/AsyncLogger.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/AsyncLogger.java (working copy) @@ -0,0 +1,264 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import java.util.Map; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; + +import org.apache.logging.log4j.Level; +import org.apache.logging.log4j.Marker; +import org.apache.logging.log4j.ThreadContext; +import org.apache.logging.log4j.core.Logger; +import org.apache.logging.log4j.core.LoggerContext; +import org.apache.logging.log4j.core.config.Property; +import org.apache.logging.log4j.core.impl.Log4jLogEvent; +import org.apache.logging.log4j.message.Message; +import org.apache.logging.log4j.message.MessageFactory; +import org.apache.logging.log4j.status.StatusLogger; + +import com.lmax.disruptor.BlockingWaitStrategy; +import com.lmax.disruptor.EventHandler; +import com.lmax.disruptor.ExceptionHandler; +import com.lmax.disruptor.RingBuffer; +import com.lmax.disruptor.SleepingWaitStrategy; +import com.lmax.disruptor.WaitStrategy; +import com.lmax.disruptor.YieldingWaitStrategy; +import com.lmax.disruptor.dsl.Disruptor; +import com.lmax.disruptor.dsl.ProducerType; +import com.lmax.disruptor.util.Util; + +/** + * AsyncLogger is a logger designed for high throughput and low latency logging. + * It does not perform any I/O in the calling (application) thread, but instead + * hands off the work to another thread as soon as possible. The actual logging + * is performed in the background thread. It uses the LMAX Disruptor library for + * inter-thread communication. (http://lmax-exchange.github.com/disruptor/) + *

+ * To use AsyncLogger, specify the System property + * {@code -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector} + * before you obtain a Logger, and all Loggers returned by LogManager.getLogger + * will be AsyncLoggers. + *

+ * Note that for performance reasons, this logger does not include source + * location by default. You need to specify {@code includeLocation="true"} in + * the configuration or any %class, %location or %line conversion patterns in + * your log4j.xml configuration will produce either a "?" character or no output + * at all. + *

+ * For best performance, use AsyncLogger with the FastFileAppender or + * FastRollingFileAppender, with immediateFlush=false. These appenders have + * built-in support for the batching mechanism used by the Disruptor library, + * and they will flush to disk at the end of each batch. This means that even + * with immediateFlush=false, there will never be any items left in the buffer; + * all log events will all be written to disk in a very efficient manner. + */ +public class AsyncLogger extends Logger { + private static final int HALF_A_SECOND = 500; + private static final int MAX_DRAIN_ATTEMPTS_BEFORE_SHUTDOWN = 20; + private static final int RINGBUFFER_MIN_SIZE = 128; + private static final int RINGBUFFER_DEFAULT_SIZE = 256 * 1024; + private static final StatusLogger LOGGER = StatusLogger.getLogger(); + + private static volatile Disruptor disruptor; + private static Clock clock = ClockFactory.getClock(); + + private static ExecutorService executor = Executors + .newSingleThreadExecutor(); + private ThreadLocal threadlocalInfo = new ThreadLocal(); + + static { + int ringBufferSize = calculateRingBufferSize(); + + WaitStrategy waitStrategy = createWaitStrategy(); + disruptor = new Disruptor( + RingBufferLogEvent.FACTORY, ringBufferSize, executor, + ProducerType.MULTI, waitStrategy); + EventHandler[] handlers = new RingBufferLogEventHandler[] {// + new RingBufferLogEventHandler() }; + disruptor.handleExceptionsWith(getExceptionHandler()); + disruptor.handleEventsWith(handlers); + + LOGGER.debug( + "Starting AsyncLogger disruptor with ringbuffer size {}...", + disruptor.getRingBuffer().getBufferSize()); + disruptor.start(); + } + + private static int calculateRingBufferSize() { + int ringBufferSize = RINGBUFFER_DEFAULT_SIZE; + String userPreferredRBSize = System.getProperty( + "AsyncLogger.RingBufferSize", String.valueOf(ringBufferSize)); + try { + int size = Integer.parseInt(userPreferredRBSize); + if (size < RINGBUFFER_MIN_SIZE) { + size = RINGBUFFER_MIN_SIZE; + LOGGER.warn( + "Invalid RingBufferSize {}, using minimum size {}.", + userPreferredRBSize, RINGBUFFER_MIN_SIZE); + } + ringBufferSize = size; + } catch (Exception ex) { + LOGGER.warn("Invalid RingBufferSize {}, using default size {}.", + userPreferredRBSize, ringBufferSize); + } + return Util.ceilingNextPowerOfTwo(ringBufferSize); + } + + private static WaitStrategy createWaitStrategy() { + String strategy = System.getProperty("AsyncLogger.WaitStrategy"); + LOGGER.debug("property AsyncLogger.WaitStrategy={}", strategy); + if ("Sleep".equals(strategy)) { + LOGGER.debug("disruptor event handler uses SleepingWaitStrategy"); + return new SleepingWaitStrategy(); + } else if ("Yield".equals(strategy)) { + LOGGER.debug("disruptor event handler uses YieldingWaitStrategy"); + return new YieldingWaitStrategy(); + } else if ("Block".equals(strategy)) { + LOGGER.debug("disruptor event handler uses BlockingWaitStrategy"); + return new BlockingWaitStrategy(); + } + LOGGER.debug("disruptor event handler uses SleepingWaitStrategy"); + return new SleepingWaitStrategy(); + } + + private static ExceptionHandler getExceptionHandler() { + String cls = System.getProperty("AsyncLogger.ExceptionHandler"); + if (cls == null) { + LOGGER.debug("No AsyncLogger.ExceptionHandler specified"); + return null; + } + try { + @SuppressWarnings("unchecked") + Class klass = (Class) Class + .forName(cls); + ExceptionHandler result = klass.newInstance(); + LOGGER.debug("AsyncLogger.ExceptionHandler=" + result); + return result; + } catch (Exception ignored) { + LOGGER.debug( + "AsyncLogger.ExceptionHandler not set: error creating " + + cls + ": ", ignored); + return null; + } + } + + /** + * Constructs an {@code AsyncLogger} with the specified context, name and + * message factory. + * + * @param context context of this logger + * @param name name of this logger + * @param messageFactory message factory of this logger + */ + public AsyncLogger(LoggerContext context, String name, + MessageFactory messageFactory) { + super(context, name, messageFactory); + } + + /** + * Tuple with the event translator and thread name for a thread. + */ + private static class Info { + private RingBufferLogEventTranslator translator; + private String cachedThreadName; + } + + @Override + public void log(Marker marker, String fqcn, Level level, Message data, + Throwable t) { + Info info = threadlocalInfo.get(); + if (info == null) { + info = new Info(); + info.translator = new RingBufferLogEventTranslator(); + info.cachedThreadName = Thread.currentThread().getName(); + threadlocalInfo.set(info); + } + + Boolean includeLocation = config.loggerConfig.isIncludeLocation(); + info.translator.setValues(this, getName(), marker, fqcn, level, data, + t, // + + // config properties are taken care of in the EventHandler + // thread in the #actualAsyncLog method + + // needs shallow copy to be fast (LOG4J2-154) + ThreadContext.getImmutableContext(), // + + // needs shallow copy to be fast (LOG4J2-154) + ThreadContext.getImmutableStack(), // + + // Thread.currentThread().getName(), // + info.cachedThreadName, // + + // location: very expensive operation. LOG4J2-153: + // Only include if "includeLocation=true" is specified, + // exclude if not specified or if "false" was specified. + includeLocation != null && includeLocation ? location(fqcn) + : null, + + // System.currentTimeMillis()); + // CoarseCachedClock: 20% faster than system clock, 16ms gaps + // CachedClock: 10% faster than system clock, smaller gaps + clock.currentTimeMillis()); + + disruptor.publishEvent(info.translator); + } + + private StackTraceElement location(String fqcnOfLogger) { + return Log4jLogEvent.calcLocation(fqcnOfLogger); + } + + /** + * This method is called by the EventHandler that processes the + * RingBufferLogEvent in a separate thread. + * + * @param event the event to log + */ + public void actualAsyncLog(RingBufferLogEvent event) { + Map properties = config.loggerConfig.getProperties(); + event.mergePropertiesIntoContextMap(properties, + config.config.getSubst()); + config.logEvent(event); + } + + public static void stop() { + Disruptor temp = disruptor; + + // Must guarantee that publishing to the RingBuffer has stopped + // before we call disruptor.shutdown() + disruptor = null; // client code fails with NPE if log after stop = OK + temp.shutdown(); + + // wait up to 10 seconds for the ringbuffer to drain + RingBuffer ringBuffer = temp.getRingBuffer(); + for (int i = 0; i < MAX_DRAIN_ATTEMPTS_BEFORE_SHUTDOWN; i++) { + if (ringBuffer.hasAvailableCapacity(ringBuffer.getBufferSize())) { + break; + } + try { + // give ringbuffer some time to drain... + Thread.sleep(HALF_A_SECOND); + } catch (InterruptedException e) { + // ignored + } + } + executor.shutdown(); // finally, kill the processor thread + } +} Index: core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerConfig.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerConfig.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerConfig.java (working copy) @@ -0,0 +1,228 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import java.util.Arrays; +import java.util.List; + +import org.apache.logging.log4j.Level; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.Filter; +import org.apache.logging.log4j.core.LogEvent; +import org.apache.logging.log4j.core.LoggerContext; +import org.apache.logging.log4j.core.LoggerContext.Status; +import org.apache.logging.log4j.core.config.AppenderRef; +import org.apache.logging.log4j.core.config.Configuration; +import org.apache.logging.log4j.core.config.LoggerConfig; +import org.apache.logging.log4j.core.config.Property; +import org.apache.logging.log4j.core.config.plugins.Plugin; +import org.apache.logging.log4j.core.config.plugins.PluginAttr; +import org.apache.logging.log4j.core.config.plugins.PluginConfiguration; +import org.apache.logging.log4j.core.config.plugins.PluginElement; +import org.apache.logging.log4j.core.config.plugins.PluginFactory; +import org.apache.logging.log4j.status.StatusLogger; + +/** + * Asynchronous Logger object that is created via configuration and can be + * combined with synchronous loggers. + *

+ * AsyncLoggerConfig is a logger designed for high throughput and low latency + * logging. It does not perform any I/O in the calling (application) thread, but + * instead hands off the work to another thread as soon as possible. The actual + * logging is performed in the background thread. It uses the LMAX Disruptor + * library for inter-thread communication. (http://lmax-exchange.github.com/disruptor/) + *

+ * To use AsyncLoggerConfig, specify {@code } or + * {@code } in configuration. + *

+ * Note that for performance reasons, this logger does not include source + * location by default. You need to specify {@code includeLocation="true"} in + * the configuration or any %class, %location or %line conversion patterns in + * your log4j.xml configuration will produce either a "?" character or no output + * at all. + *

+ * For best performance, use AsyncLoggerConfig with the FastFileAppender or + * FastRollingFileAppender, with immediateFlush=false. These appenders have + * built-in support for the batching mechanism used by the Disruptor library, + * and they will flush to disk at the end of each batch. This means that even + * with immediateFlush=false, there will never be any items left in the buffer; + * all log events will all be written to disk in a very efficient manner. + */ +@Plugin(name = "asyncLogger", type = "Core", printObject = true) +public class AsyncLoggerConfig extends LoggerConfig { + + private static final Logger LOGGER = StatusLogger.getLogger(); + private AsyncLoggerConfigHelper helper; + + /** + * Default constructor. + */ + public AsyncLoggerConfig() { + super(); + } + + /** + * Constructor that sets the name, level and additive values. + * + * @param name The Logger name. + * @param level The Level. + * @param additive true if the Logger is additive, false otherwise. + */ + public AsyncLoggerConfig(final String name, final Level level, + final boolean additive) { + super(name, level, additive); + } + + protected AsyncLoggerConfig(final String name, + final List appenders, final Filter filter, + final Level level, final boolean additive, + final Property[] properties, final Configuration config, + final boolean includeLocation) { + super(name, appenders, filter, level, additive, properties, config, + includeLocation); + } + + /** + * Passes on the event to a separate thread that will call + * {@link #asyncCallAppenders(LogEvent)}. + */ + @Override + protected void callAppenders(LogEvent event) { + // populate lazily initialized fields + event.getSource(); + event.getThreadName(); + + // pass on the event to a separate thread + helper.callAppendersFromAnotherThread(event); + } + + /** Called by AsyncLoggerConfigHelper.RingBufferLog4jEventHandler. */ + void asyncCallAppenders(LogEvent event) { + super.callAppenders(event); + } + + @Override + public void startFilter() { + if (helper == null) { + helper = new AsyncLoggerConfigHelper(this); + } + super.startFilter(); + } + + @Override + public void stopFilter() { + // only stop disruptor if shutting down logging subsystem + if (LogManager.getContext() instanceof LoggerContext) { + if (((LoggerContext) LogManager.getContext()).getStatus() != Status.STOPPING) { + return; + } + } + helper.shutdown(); + super.stopFilter(); + } + + /** + * Factory method to create a LoggerConfig. + * + * @param additivity True if additive, false otherwise. + * @param levelName The Level to be associated with the Logger. + * @param loggerName The name of the Logger. + * @param includeLocation "true" if location should be passed downstream + * @param refs An array of Appender names. + * @param properties Properties to pass to the Logger. + * @param config The Configuration. + * @param filter A Filter. + * @return A new LoggerConfig. + */ + @PluginFactory + public static LoggerConfig createLogger( + @PluginAttr("additivity") final String additivity, + @PluginAttr("level") final String levelName, + @PluginAttr("name") final String loggerName, + @PluginAttr("includeLocation") final String includeLocation, + @PluginElement("appender-ref") final AppenderRef[] refs, + @PluginElement("properties") final Property[] properties, + @PluginConfiguration final Configuration config, + @PluginElement("filters") final Filter filter) { + if (loggerName == null) { + LOGGER.error("Loggers cannot be configured without a name"); + return null; + } + + final List appenderRefs = Arrays.asList(refs); + Level level; + try { + level = Level.toLevel(levelName, Level.ERROR); + } catch (final Exception ex) { + LOGGER.error( + "Invalid Log level specified: {}. Defaulting to Error", + levelName); + level = Level.ERROR; + } + final String name = loggerName.equals("root") ? "" : loggerName; + final boolean additive = additivity == null ? true : Boolean + .parseBoolean(additivity); + + return new AsyncLoggerConfig(name, appenderRefs, filter, level, + additive, properties, config, includeLocation(includeLocation)); + } + + // Note: for asynchronous loggers, includeLocation default is FALSE + private static boolean includeLocation(String includeLocationConfigValue) { + if (includeLocationConfigValue == null) { + return false; + } + return Boolean.parseBoolean(includeLocationConfigValue); + } + + /** + * An asynchronous root Logger. + */ + @Plugin(name = "asyncRoot", type = "Core", printObject = true) + public static class RootLogger extends LoggerConfig { + + @PluginFactory + public static LoggerConfig createLogger( + @PluginAttr("additivity") final String additivity, + @PluginAttr("level") final String levelName, + @PluginAttr("includeLocation") final String includeLocation, + @PluginElement("appender-ref") final AppenderRef[] refs, + @PluginElement("properties") final Property[] properties, + @PluginConfiguration final Configuration config, + @PluginElement("filters") final Filter filter) { + final List appenderRefs = Arrays.asList(refs); + Level level; + try { + level = Level.toLevel(levelName, Level.ERROR); + } catch (final Exception ex) { + LOGGER.error( + "Invalid Log level specified: {}. Defaulting to Error", + levelName); + level = Level.ERROR; + } + final boolean additive = additivity == null ? true : Boolean + .parseBoolean(additivity); + + return new AsyncLoggerConfig(LogManager.ROOT_LOGGER_NAME, + appenderRefs, filter, level, additive, properties, config, + includeLocation(includeLocation)); + } + } +} Index: core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerConfigHelper.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerConfigHelper.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerConfigHelper.java (working copy) @@ -0,0 +1,245 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; + +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LogEvent; +import org.apache.logging.log4j.status.StatusLogger; + +import com.lmax.disruptor.BlockingWaitStrategy; +import com.lmax.disruptor.EventFactory; +import com.lmax.disruptor.EventHandler; +import com.lmax.disruptor.EventTranslator; +import com.lmax.disruptor.ExceptionHandler; +import com.lmax.disruptor.RingBuffer; +import com.lmax.disruptor.Sequence; +import com.lmax.disruptor.SequenceReportingEventHandler; +import com.lmax.disruptor.SleepingWaitStrategy; +import com.lmax.disruptor.WaitStrategy; +import com.lmax.disruptor.YieldingWaitStrategy; +import com.lmax.disruptor.dsl.Disruptor; +import com.lmax.disruptor.dsl.ProducerType; +import com.lmax.disruptor.util.Util; + +/** + * Helper class decoupling the {@code AsyncLoggerConfig} class from the LMAX + * Disruptor library. + *

+ * {@code AsyncLoggerConfig} is a plugin, and will be loaded even if users do + * not configure any {@code } or {@code } elements in + * the configuration. If {@code AsyncLoggerConfig} has inner classes that extend + * or implement classes from the Disruptor library, a + * {@code NoClassDefFoundError} is thrown if the Disruptor jar is not in the + * classpath when the PluginManager loads the {@code AsyncLoggerConfig} plugin + * from the pre-defined plugins definition file. + *

+ * This class serves to make the dependency on the Disruptor optional, so that + * these classes are only loaded when the {@code AsyncLoggerConfig} is actually + * used. + */ +class AsyncLoggerConfigHelper { + + private static final int MAX_DRAIN_ATTEMPTS_BEFORE_SHUTDOWN = 20; + private static final int HALF_A_SECOND = 500; + private static final int RINGBUFFER_MIN_SIZE = 128; + private static final int RINGBUFFER_DEFAULT_SIZE = 256 * 1024; + private static final Logger LOGGER = StatusLogger.getLogger(); + + private static volatile Disruptor disruptor; + private static ExecutorService executor = Executors + .newSingleThreadExecutor(); + + /** + * Factory used to populate the RingBuffer with events. These event objects + * are then re-used during the life of the RingBuffer. + */ + private static final EventFactory FACTORY = new EventFactory() { + @Override + public RingBufferLog4jEvent newInstance() { + return new RingBufferLog4jEvent(); + } + }; + + /** + * Object responsible for passing on data to a specific RingBuffer event. + */ + private final EventTranslator translator = new EventTranslator() { + @Override + public void translateTo(RingBufferLog4jEvent event, long sequence) { + event.event = currentLogEvent.get(); + event.loggerConfig = asyncLoggerConfig; + } + }; + + private ThreadLocal currentLogEvent = new ThreadLocal(); + private AsyncLoggerConfig asyncLoggerConfig; + + public AsyncLoggerConfigHelper(AsyncLoggerConfig asyncLoggerConfig) { + this.asyncLoggerConfig = asyncLoggerConfig; + initDisruptor(); + } + + private static synchronized void initDisruptor() { + if (disruptor != null) { + return; + } + int ringBufferSize = calculateRingBufferSize(); + WaitStrategy waitStrategy = createWaitStrategy(); + disruptor = new Disruptor(FACTORY, + ringBufferSize, executor, ProducerType.MULTI, waitStrategy); + EventHandler[] handlers = new RingBufferLog4jEventHandler[] {// + new RingBufferLog4jEventHandler() }; + disruptor.handleExceptionsWith(getExceptionHandler()); + disruptor.handleEventsWith(handlers); + + LOGGER.debug( + "Starting AsyncLoggerConfig disruptor with ringbuffer size {}...", + disruptor.getRingBuffer().getBufferSize()); + disruptor.start(); + } + + private static WaitStrategy createWaitStrategy() { + String strategy = System.getProperty("AsyncLoggerConfig.WaitStrategy"); + LOGGER.debug("property AsyncLoggerConfig.WaitStrategy={}", strategy); + if ("Sleep".equals(strategy)) { + LOGGER.debug("disruptor event handler uses SleepingWaitStrategy"); + return new SleepingWaitStrategy(); + } else if ("Yield".equals(strategy)) { + LOGGER.debug("disruptor event handler uses YieldingWaitStrategy"); + return new YieldingWaitStrategy(); + } else if ("Block".equals(strategy)) { + LOGGER.debug("disruptor event handler uses BlockingWaitStrategy"); + return new BlockingWaitStrategy(); + } + LOGGER.debug("disruptor event handler uses SleepingWaitStrategy"); + return new SleepingWaitStrategy(); + } + + private static int calculateRingBufferSize() { + int ringBufferSize = RINGBUFFER_DEFAULT_SIZE; + String userPreferredRBSize = System.getProperty( + "AsyncLoggerConfig.RingBufferSize", + String.valueOf(ringBufferSize)); + try { + int size = Integer.parseInt(userPreferredRBSize); + if (size < RINGBUFFER_MIN_SIZE) { + size = RINGBUFFER_MIN_SIZE; + LOGGER.warn( + "Invalid RingBufferSize {}, using minimum size {}.", + userPreferredRBSize, RINGBUFFER_MIN_SIZE); + } + ringBufferSize = size; + } catch (Exception ex) { + LOGGER.warn("Invalid RingBufferSize {}, using default size {}.", + userPreferredRBSize, ringBufferSize); + } + return Util.ceilingNextPowerOfTwo(ringBufferSize); + } + + private static ExceptionHandler getExceptionHandler() { + String cls = System.getProperty("AsyncLoggerConfig.ExceptionHandler"); + if (cls == null) { + LOGGER.debug("No AsyncLoggerConfig.ExceptionHandler specified"); + return null; + } + try { + @SuppressWarnings("unchecked") + Class klass = (Class) Class + .forName(cls); + ExceptionHandler result = klass.newInstance(); + LOGGER.debug("AsyncLoggerConfig.ExceptionHandler=" + result); + return result; + } catch (Exception ignored) { + LOGGER.debug( + "AsyncLoggerConfig.ExceptionHandler not set: error creating " + + cls + ": ", ignored); + return null; + } + } + + /** + * RingBuffer events contain all information necessary to perform the work + * in a separate thread. + */ + private static class RingBufferLog4jEvent { + private AsyncLoggerConfig loggerConfig; + private LogEvent event; + } + + /** + * EventHandler performs the work in a separate thread. + */ + private static class RingBufferLog4jEventHandler implements + SequenceReportingEventHandler { + private static final int NOTIFY_PROGRESS_THRESHOLD = 50; + private Sequence sequenceCallback; + private int counter; + + @Override + public void setSequenceCallback(Sequence sequenceCallback) { + this.sequenceCallback = sequenceCallback; + } + + @Override + public void onEvent(RingBufferLog4jEvent event, long sequence, + boolean endOfBatch) throws Exception { + event.event.setEndOfBatch(endOfBatch); + event.loggerConfig.asyncCallAppenders(event.event); + + // notify the BatchEventProcessor that the sequence has progressed. + // Without this callback the sequence would not be progressed + // until the batch has completely finished. + if (++counter > NOTIFY_PROGRESS_THRESHOLD) { + sequenceCallback.set(sequence); + counter = 0; + } + } + } + + public void shutdown() { + Disruptor temp = disruptor; + + // Must guarantee that publishing to the RingBuffer has stopped + // before we call disruptor.shutdown() + disruptor = null; // client code fails with NPE if log after stop = OK + temp.shutdown(); + + // wait up to 10 seconds for the ringbuffer to drain + RingBuffer ringBuffer = temp.getRingBuffer(); + for (int i = 0; i < MAX_DRAIN_ATTEMPTS_BEFORE_SHUTDOWN; i++) { + if (ringBuffer.hasAvailableCapacity(ringBuffer.getBufferSize())) { + break; + } + try { + // give ringbuffer some time to drain... + Thread.sleep(HALF_A_SECOND); + } catch (InterruptedException e) { + // ignored + } + } + executor.shutdown(); // finally, kill the processor thread + } + + public void callAppendersFromAnotherThread(LogEvent event) { + currentLogEvent.set(event); + disruptor.publishEvent(translator); + } + +} Index: core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerContext.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerContext.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerContext.java (working copy) @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import java.net.URI; + +import org.apache.logging.log4j.core.Logger; +import org.apache.logging.log4j.core.LoggerContext; +import org.apache.logging.log4j.message.MessageFactory; + +/** + * {@code LoggerContext} that creates {@code AsyncLogger} objects. + */ +public class AsyncLoggerContext extends LoggerContext { + + public AsyncLoggerContext(String name) { + super(name); + } + + public AsyncLoggerContext(String name, Object externalContext) { + super(name, externalContext); + } + + public AsyncLoggerContext(String name, Object externalContext, + URI configLocn) { + super(name, externalContext, configLocn); + } + + public AsyncLoggerContext(String name, Object externalContext, + String configLocn) { + super(name, externalContext, configLocn); + } + + @Override + protected Logger newInstance(LoggerContext ctx, String name, + MessageFactory messageFactory) { + return new AsyncLogger(ctx, name, messageFactory); + } + + @Override + public void stop() { + AsyncLogger.stop(); + super.stop(); + } +} Index: core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerContextSelector.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerContextSelector.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/AsyncLoggerContextSelector.java (working copy) @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import java.net.URI; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +import org.apache.logging.log4j.core.LoggerContext; +import org.apache.logging.log4j.core.selector.ContextSelector; + +/** + * {@code ContextSelector} that returns the singleton {@code AsyncLoggerContext}. + */ +public class AsyncLoggerContextSelector implements ContextSelector { + + private static final AsyncLoggerContext CONTEXT = new AsyncLoggerContext( + "AsyncLoggerContext"); + + public LoggerContext getContext(String fqcn, ClassLoader loader, + boolean currentContext) { + return CONTEXT; + } + + public List getLoggerContexts() { + List list = new ArrayList(); + list.add(CONTEXT); + return Collections.unmodifiableList(list); + } + + public LoggerContext getContext(String fqcn, ClassLoader loader, + boolean currentContext, URI configLocation) { + return CONTEXT; + } + +} Index: core/src/main/java/org/apache/logging/log4j/core/async/CachedClock.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/CachedClock.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/CachedClock.java (working copy) @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import com.lmax.disruptor.util.Util; + +/** + * Implementation of the {@code Clock} interface that tracks the time in a + * private long field that is updated by a background thread once every + * millisecond. Timers on most platforms do not have millisecond granularity, so + * the returned value may "jump" every 10 or 16 milliseconds. To reduce this + * problem, this class also updates the internal time value every 1024 calls to + * {@code currentTimeMillis()}. + */ +public final class CachedClock implements Clock { + private static final int UPDATE_THRESHOLD = 0x3FF; + private static CachedClock instance = new CachedClock(); + private volatile long millis = System.currentTimeMillis(); + private volatile short count = 0; + private final Thread updater = new Thread("Clock Updater Thread") { + public void run() { + while (true) { + long time = System.currentTimeMillis(); + millis = time; + Util.getUnsafe().park(true, time + 1); // abs (millis) + // Util.getUnsafe().park(false, 1000 * 1000);// relative(nanos) + } + } + }; + + private CachedClock() { + updater.setDaemon(true); + updater.start(); + } + + public static CachedClock instance() { + return instance; + } + + /** + * Returns the value of a private long field that is updated by a background + * thread once every millisecond. Timers on most platforms do not + * have millisecond granularity, the returned value may "jump" every 10 or + * 16 milliseconds. To reduce this problem, this method also updates the + * internal time value every 1024 calls. + * @return the cached time + */ + @Override + public long currentTimeMillis() { + + // improve granularity: also update time field every 1024 calls. + // (the bit fiddling means we don't need to worry about overflows) + if ((++count & UPDATE_THRESHOLD) == UPDATE_THRESHOLD) { + millis = System.currentTimeMillis(); + } + return millis; + } +} Index: core/src/main/java/org/apache/logging/log4j/core/async/Clock.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/Clock.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/Clock.java (working copy) @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +/** + * Interface for classes that can provide the time stamp used in log events. + */ +public interface Clock { + /** + * Returns the time in milliseconds since the epoch. + * + * @return the time in milliseconds since the epoch + */ + long currentTimeMillis(); +} Index: core/src/main/java/org/apache/logging/log4j/core/async/ClockFactory.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/ClockFactory.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/ClockFactory.java (working copy) @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import org.apache.logging.log4j.status.StatusLogger; + +/** + * Factory for {@code Clock} objects. + */ +public final class ClockFactory { + + /** + * Name of the system property that can be used to specify a {@code Clock} + * implementation class. + */ + public static final String PROPERTY_NAME = "AsyncLogger.Clock"; + private static final StatusLogger LOGGER = StatusLogger.getLogger(); + + // private static final Clock clock = createClock(); + + private ClockFactory() { + } + + /** + * Returns a {@code Clock} instance depending on the value of system + * property {@code "AsyncLogger.Clock"}. + *

+ * If system property {@code AsyncLogger.Clock=CachedClock} is specified, + * this method returns an instance of {@link CachedClock}. If system + * property {@code AsyncLogger.Clock=CoarseCachedClock} is specified, this + * method returns an instance of {@link CoarseCachedClock}. + *

+ * If another value is specified, this value is taken as the fully qualified + * class name of a class that implements the {@code Clock} interface. An + * object of this class is instantiated and returned. + *

+ * If no value is specified, or if the specified value could not correctly + * be instantiated or did not implement the {@code Clock} interface, then an + * instance of {@link SystemClock} is returned. + * + * @return a {@code Clock} instance + */ + public static Clock getClock() { + return createClock(); + } + + private static Clock createClock() { + String userRequest = System.getProperty(PROPERTY_NAME); + if (userRequest == null || "SystemClock".equals(userRequest)) { + LOGGER.debug("Using default SystemClock for timestamps"); + return new SystemClock(); + } + if (CachedClock.class.getName().equals(userRequest) // + || "CachedClock".equals(userRequest)) { + LOGGER.debug("Using specified CachedClock for timestamps"); + return CachedClock.instance(); + } + if (CoarseCachedClock.class.getName().equals(userRequest) // + || "CoarseCachedClock".equals(userRequest)) { + LOGGER.debug("Using specified CoarseCachedClock for timestamps"); + return CoarseCachedClock.instance(); + } + try { + Clock result = (Clock) Class.forName(userRequest).newInstance(); + LOGGER.debug("Using {} for timestamps", userRequest); + return result; + } catch (Exception e) { + String fmt = "Could not create {}: {}, using default SystemClock for timestamps"; + LOGGER.error(fmt, userRequest, e); + return new SystemClock(); + } + } +} Index: core/src/main/java/org/apache/logging/log4j/core/async/CoarseCachedClock.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/CoarseCachedClock.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/CoarseCachedClock.java (working copy) @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import com.lmax.disruptor.util.Util; + +/** + * This Clock implementation is similar to CachedClock. It is slightly faster at + * the cost of some accuracy. + */ +public final class CoarseCachedClock implements Clock { + private static CoarseCachedClock instance = new CoarseCachedClock(); + private volatile long millis = System.currentTimeMillis(); + + private final Thread updater = new Thread("Clock Updater Thread") { + public void run() { + while (true) { + long time = System.currentTimeMillis(); + millis = time; + Util.getUnsafe().park(true, time + 1); // abs (millis) + // Util.getUnsafe().park(false, 1000 * 1000);// relative(nanos) + } + } + }; + + private CoarseCachedClock() { + updater.setDaemon(true); + updater.start(); + } + + /** + * Returns the singleton instance. + * + * @return the singleton instance + */ + public static CoarseCachedClock instance() { + return instance; + } + + /** + * Returns the value of a private long field that is updated by a background + * thread once every millisecond. Because timers on most platforms do not + * have millisecond granularity, the returned value may "jump" every 10 or + * 16 milliseconds. + * @return the cached time + */ + @Override + public long currentTimeMillis() { + return millis; + } +} Index: core/src/main/java/org/apache/logging/log4j/core/async/RingBufferLogEvent.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/RingBufferLogEvent.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/RingBufferLogEvent.java (working copy) @@ -0,0 +1,208 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import java.util.HashMap; +import java.util.Map; + +import org.apache.logging.log4j.Level; +import org.apache.logging.log4j.Marker; +import org.apache.logging.log4j.ThreadContext.ContextStack; +import org.apache.logging.log4j.core.LogEvent; +import org.apache.logging.log4j.core.config.Property; +import org.apache.logging.log4j.core.lookup.StrSubstitutor; +import org.apache.logging.log4j.message.Message; +import org.apache.logging.log4j.message.SimpleMessage; + +import com.lmax.disruptor.EventFactory; + +/** + * When the Disruptor is started, the RingBuffer is populated with event + * objects. These objects are then re-used during the life of the RingBuffer. + */ +public class RingBufferLogEvent implements LogEvent { + private static final long serialVersionUID = 8462119088943934758L; + + /** + * Creates the events that will be put in the RingBuffer. + */ + private static class Factory implements EventFactory { + // @Override + public RingBufferLogEvent newInstance() { + return new RingBufferLogEvent(); + } + } + + /** The {@code EventFactory} for {@code RingBufferLogEvent}s. */ + public static final Factory FACTORY = new Factory(); + + private AsyncLogger asyncLogger; + private String loggerName; + private Marker marker; + private String fqcn; + private Level level; + private Message message; + private Throwable thrown; + private Map contextMap; + private ContextStack contextStack; + private String threadName; + private StackTraceElement location; + private long currentTimeMillis; + private boolean endOfBatch; + private boolean includeLocation; + + public void setValues(AsyncLogger asyncLogger, String loggerName, + Marker marker, String fqcn, Level level, Message data, Throwable t, + Map map, ContextStack contextStack, + String threadName, StackTraceElement location, + long currentTimeMillis) { + this.asyncLogger = asyncLogger; + this.loggerName = loggerName; + this.marker = marker; + this.fqcn = fqcn; + this.level = level; + this.message = data; + this.thrown = t; + this.contextMap = map; + this.contextStack = contextStack; + this.threadName = threadName; + this.location = location; + this.currentTimeMillis = currentTimeMillis; + } + + /** + * Event processor that reads the event from the ringbuffer can call this + * method. + * + * @param endOfBatch flag to indicate if this is the last event in a batch + * from the RingBuffer + */ + public void execute(boolean endOfBatch) { + this.endOfBatch = endOfBatch; + asyncLogger.actualAsyncLog(this); + } + + /** + * Returns {@code true} if this event is the end of a batch, {@code false} + * otherwise. + * + * @return {@code true} if this event is the end of a batch, {@code false} + * otherwise + */ + public boolean isEndOfBatch() { + return endOfBatch; + } + + public void setEndOfBatch(boolean endOfBatch) { + this.endOfBatch = endOfBatch; + } + + public boolean isIncludeLocation() { + return includeLocation; + } + + public void setIncludeLocation(boolean includeLocation) { + this.includeLocation = includeLocation; + } + + // @Override + public String getLoggerName() { + return loggerName; + } + + // @Override + public Marker getMarker() { + return marker; + } + + // @Override + public String getFQCN() { + return fqcn; + } + + // @Override + public Level getLevel() { + return level; + } + + // @Override + public Message getMessage() { + if (message == null) { + message = new SimpleMessage(""); + } + return message; + } + + // @Override + public Throwable getThrown() { + return thrown; + } + + // @Override + public Map getContextMap() { + return contextMap; + } + + // @Override + public ContextStack getContextStack() { + return contextStack; + } + + // @Override + public String getThreadName() { + return threadName; + } + + // @Override + public StackTraceElement getSource() { + return location; + } + + // @Override + public long getMillis() { + return currentTimeMillis; + } + + /** + * Merges the contents of the specified map into the contextMap, after + * replacing any variables in the property values with the + * StrSubstitutor-supplied actual values. + * + * @param properties configured properties + * @param strSubstitutor used to lookup values of variables in properties + */ + public void mergePropertiesIntoContextMap( + Map properties, StrSubstitutor strSubstitutor) { + if (properties == null) { + return; // nothing to do + } + + Map map = (contextMap == null) ? new HashMap() + : new HashMap(contextMap); + + for (Map.Entry entry : properties.entrySet()) { + Property prop = entry.getKey(); + if (map.containsKey(prop.getName())) { + continue; // contextMap overrides config properties + } + String value = entry.getValue() ? strSubstitutor.replace(prop + .getValue()) : prop.getValue(); + map.put(prop.getName(), value); + } + contextMap = map; + } +} Index: core/src/main/java/org/apache/logging/log4j/core/async/RingBufferLogEventHandler.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/RingBufferLogEventHandler.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/RingBufferLogEventHandler.java (working copy) @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import com.lmax.disruptor.Sequence; +import com.lmax.disruptor.SequenceReportingEventHandler; + +/** + * This event handler gets passed messages from the RingBuffer as they become + * available. Processing of these messages is done in a separate thread, + * controlled by the {@code Executor} passed to the {@code Disruptor} + * constructor. + */ +public class RingBufferLogEventHandler implements + SequenceReportingEventHandler { + + private static final int NOTIFY_PROGRESS_THRESHOLD = 50; + private Sequence sequenceCallback; + private int counter; + + @Override + public void setSequenceCallback(Sequence sequenceCallback) { + this.sequenceCallback = sequenceCallback; + } + + @Override + public void onEvent(RingBufferLogEvent event, long sequence, + boolean endOfBatch) throws Exception { + event.execute(endOfBatch); + + // notify the BatchEventProcessor that the sequence has progressed. + // Without this callback the sequence would not be progressed + // until the batch has completely finished. + if (++counter > NOTIFY_PROGRESS_THRESHOLD) { + sequenceCallback.set(sequence); + counter = 0; + } + } + +} Index: core/src/main/java/org/apache/logging/log4j/core/async/RingBufferLogEventTranslator.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/RingBufferLogEventTranslator.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/RingBufferLogEventTranslator.java (working copy) @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import java.util.Map; + +import org.apache.logging.log4j.Level; +import org.apache.logging.log4j.Marker; +import org.apache.logging.log4j.ThreadContext.ContextStack; +import org.apache.logging.log4j.message.Message; + +import com.lmax.disruptor.EventTranslator; + +/** + * This class is responsible for writing elements that make up a log event into + * the ringbuffer {@code RingBufferLogEvent}. After this translator populated + * the ringbuffer event, the disruptor will update the sequence number so that + * the event can be consumed by another thread. + */ +public class RingBufferLogEventTranslator implements + EventTranslator { + + private AsyncLogger asyncLogger; + private String loggerName; + private Marker marker; + private String fqcn; + private Level level; + private Message message; + private Throwable thrown; + private Map contextMap; + private ContextStack contextStack; + private String threadName; + private StackTraceElement location; + private long currentTimeMillis; + + // @Override + public void translateTo(RingBufferLogEvent event, long sequence) { + event.setValues(asyncLogger, loggerName, marker, fqcn, level, message, + thrown, contextMap, contextStack, threadName, location, + currentTimeMillis); + } + + public void setValues(AsyncLogger asyncLogger, String loggerName, + Marker marker, String fqcn, Level level, Message message, + Throwable thrown, Map contextMap, + ContextStack contextStack, String threadName, + StackTraceElement location, long currentTimeMillis) { + this.asyncLogger = asyncLogger; + this.loggerName = loggerName; + this.marker = marker; + this.fqcn = fqcn; + this.level = level; + this.message = message; + this.thrown = thrown; + this.contextMap = contextMap; + this.contextStack = contextStack; + this.threadName = threadName; + this.location = location; + this.currentTimeMillis = currentTimeMillis; + } + +} Index: core/src/main/java/org/apache/logging/log4j/core/async/SystemClock.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/SystemClock.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/SystemClock.java (working copy) @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +/** + * Implementation of the {@code Clock} interface that returns the system time. + */ +public class SystemClock implements Clock { + + /** + * Returns the system time. + * @return the result of calling {@code System.currentTimeMillis()} + */ + @Override + public long currentTimeMillis() { + return System.currentTimeMillis(); + } + +} Index: core/src/main/java/org/apache/logging/log4j/core/async/package-info.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/async/package-info.java (revision 0) +++ core/src/main/java/org/apache/logging/log4j/core/async/package-info.java (working copy) @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +/** + * Provides Asynchronous Logger classes and interfaces for low-latency logging. + */ +package org.apache.logging.log4j.core.async; Index: core/src/main/java/org/apache/logging/log4j/core/config/LoggerConfig.java =================================================================== --- core/src/main/java/org/apache/logging/log4j/core/config/LoggerConfig.java (revision 1467676) +++ core/src/main/java/org/apache/logging/log4j/core/config/LoggerConfig.java (working copy) @@ -23,6 +23,7 @@ import org.apache.logging.log4j.core.Appender; import org.apache.logging.log4j.core.Filter; import org.apache.logging.log4j.core.LifeCycle; +import org.apache.logging.log4j.core.async.AsyncLoggerContextSelector; import org.apache.logging.log4j.core.config.plugins.PluginConfiguration; import org.apache.logging.log4j.core.filter.AbstractFilterable; import org.apache.logging.log4j.core.helpers.Constants; @@ -477,7 +478,7 @@ // for synchronous loggers, includeLocation default is TRUE. private static boolean includeLocation(String includeLocationConfigValue) { if (includeLocationConfigValue == null) { - final boolean sync = !"org.apache.logging.log4j.async.AsyncLoggerContextSelector" + final boolean sync = !AsyncLoggerContextSelector.class.getName() .equals(System.getProperty(Constants.LOG4J_CONTEXT_SELECTOR)); return sync; } Index: core/src/site/xdoc/index.xml =================================================================== --- core/src/site/xdoc/index.xml (revision 1467676) +++ core/src/site/xdoc/index.xml (working copy) @@ -26,18 +26,27 @@

- The Log4Jj 2 implementation provides the functional components of the logging system. Users are - free to create their own plugins and include them in the logging configuration. + The Log4Jj 2 implementation provides the functional components + of the logging system. + Users are free to create their own plugins and include them + in the logging configuration.

- Log4j 2 requires Java 5 but has will take advantage of enhancements in Java 6 to improve performance. - Some features may require optional dependencies. These dependencies are + Log4j 2 requires Java 6. + Some features may require optional + dependencies. These dependencies are specified in the documentation for those features.

+
    +
  • JSON configuration requires the Jackson JSON-processor.
  • +
  • Async Loggers require the LMAX Disruptor.
  • +
  • SMTPAppender requires Javax Mail.
  • +
  • JMSQueueAppender and JMSTopicAppender require a JMS implementation like geronimo-jms.
  • +
Index: core/src/test/java/org/apache/logging/log4j/core/appender/FastFileAppenderLocationTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/appender/FastFileAppenderLocationTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/appender/FastFileAppenderLocationTest.java (working copy) @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.appender; + +import static org.junit.Assert.*; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LifeCycle; +import org.apache.logging.log4j.core.config.XMLConfigurationFactory; +import org.junit.BeforeClass; +import org.junit.Test; + +public class FastFileAppenderLocationTest { + + @BeforeClass + public static void beforeClass() { + System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, + "FastFileAppenderLocationTest.xml"); + } + + @Test + public void testLocationIncluded() throws Exception { + File f = new File("target", "FastFileAppenderLocationTest.log"); + // System.out.println(f.getAbsolutePath()); + f.delete(); + Logger log = LogManager.getLogger("com.foo.Bar"); + String msg = "Message with location, flushed with immediate flush=false"; + log.info(msg); + ((LifeCycle) LogManager.getContext()).stop(); // stop async thread + + BufferedReader reader = new BufferedReader(new FileReader(f)); + String line1 = reader.readLine(); + reader.close(); + f.delete(); + assertNotNull("line1", line1); + assertTrue("line1 correct", line1.contains(msg)); + + String location = "testLocationIncluded"; + assertTrue("has location", line1.contains(location)); + } +} Index: core/src/test/java/org/apache/logging/log4j/core/appender/FastFileAppenderTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/appender/FastFileAppenderTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/appender/FastFileAppenderTest.java (working copy) @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.appender; + +import static org.junit.Assert.*; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LifeCycle; +import org.apache.logging.log4j.core.config.XMLConfigurationFactory; +import org.junit.BeforeClass; +import org.junit.Test; + +public class FastFileAppenderTest { + + @BeforeClass + public static void beforeClass() { + System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, + "FastFileAppenderTest.xml"); + } + + @Test + public void testFlushAtEndOfBatch() throws Exception { + File f = new File("target", "FastFileAppenderTest.log"); + // System.out.println(f.getAbsolutePath()); + f.delete(); + Logger log = LogManager.getLogger("com.foo.Bar"); + String msg = "Message flushed with immediate flush=false"; + log.info(msg); + ((LifeCycle) LogManager.getContext()).stop(); // stop async thread + + BufferedReader reader = new BufferedReader(new FileReader(f)); + String line1 = reader.readLine(); + reader.close(); + f.delete(); + assertNotNull("line1", line1); + assertTrue("line1 correct", line1.contains(msg)); + + String location = "testFlushAtEndOfBatch"; + assertTrue("no location", !line1.contains(location)); + } +} Index: core/src/test/java/org/apache/logging/log4j/core/appender/FastRollingFileAppenderLocationTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/appender/FastRollingFileAppenderLocationTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/appender/FastRollingFileAppenderLocationTest.java (working copy) @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.appender; + +import static org.junit.Assert.*; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LifeCycle; +import org.apache.logging.log4j.core.config.XMLConfigurationFactory; +import org.junit.BeforeClass; +import org.junit.Test; + +public class FastRollingFileAppenderLocationTest { + + @BeforeClass + public static void beforeClass() { + System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, + "FastRollingFileAppenderLocationTest.xml"); + } + + @Test + public void testLocationIncluded() throws Exception { + File f = new File("target", "FastRollingFileAppenderLocationTest.log"); + // System.out.println(f.getAbsolutePath()); + f.delete(); + Logger log = LogManager.getLogger("com.foo.Bar"); + String msg = "Message with location, flushed with immediate flush=false"; + log.info(msg); + ((LifeCycle) LogManager.getContext()).stop(); // stop async thread + + BufferedReader reader = new BufferedReader(new FileReader(f)); + String line1 = reader.readLine(); + reader.close(); + f.delete(); + assertNotNull("line1", line1); + assertTrue("line1 correct", line1.contains(msg)); + + String location = "testLocationIncluded"; + assertTrue("has location", line1.contains(location)); + } +} Index: core/src/test/java/org/apache/logging/log4j/core/appender/FastRollingFileAppenderRolloverTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/appender/FastRollingFileAppenderRolloverTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/appender/FastRollingFileAppenderRolloverTest.java (working copy) @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.appender; + +import static org.junit.Assert.*; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; +import java.util.Arrays; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LifeCycle; +import org.apache.logging.log4j.core.config.XMLConfigurationFactory; +import org.junit.BeforeClass; +import org.junit.Test; + +public class FastRollingFileAppenderRolloverTest { + + @BeforeClass + public static void beforeClass() { + System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, + "FastRollingFileAppenderTest.xml"); + } + + @Test + public void testRollover() throws Exception { + File f = new File("target", "FastRollingFileAppenderTest.log"); + // System.out.println(f.getAbsolutePath()); + File after1 = new File("target", "afterRollover-1.log"); + f.delete(); + after1.delete(); + + Logger log = LogManager.getLogger("com.foo.Bar"); + String msg = "First a short message that does not trigger rollover"; + log.info(msg); + Thread.sleep(50); + + BufferedReader reader = new BufferedReader(new FileReader(f)); + String line1 = reader.readLine(); + assertTrue(line1.contains(msg)); + reader.close(); + + assertFalse("afterRollover-1.log not created yet", after1.exists()); + + String exceed = "Long message that exceeds rollover size... "; + char[] padding = new char[250]; + Arrays.fill(padding, 'X'); + exceed += new String(padding); + log.warn(exceed); + assertFalse("exceeded size but afterRollover-1.log not created yet", after1.exists()); + + String trigger = "This message triggers rollover."; + log.warn(trigger); + + ((LifeCycle) LogManager.getContext()).stop(); // stop async thread + + assertTrue("afterRollover-1.log created", after1.exists()); + + reader = new BufferedReader(new FileReader(f)); + String new1 = reader.readLine(); + assertTrue("after rollover only new msg", new1.contains(trigger)); + assertNull("No more lines", reader.readLine()); + reader.close(); + f.delete(); + + reader = new BufferedReader(new FileReader(after1)); + String old1 = reader.readLine(); + assertTrue("renamed file line 1", old1.contains(msg)); + String old2 = reader.readLine(); + assertTrue("renamed file line 2", old2.contains(exceed)); + String line = reader.readLine(); + assertNull("No more lines", line); + reader.close(); + after1.delete(); + } +} Index: core/src/test/java/org/apache/logging/log4j/core/appender/FastRollingFileAppenderTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/appender/FastRollingFileAppenderTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/appender/FastRollingFileAppenderTest.java (working copy) @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.appender; + +import static org.junit.Assert.*; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LifeCycle; +import org.apache.logging.log4j.core.config.XMLConfigurationFactory; +import org.junit.BeforeClass; +import org.junit.Test; + +public class FastRollingFileAppenderTest { + + @BeforeClass + public static void beforeClass() { + System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, + "FastRollingFileAppenderTest.xml"); + } + + @Test + public void testFlushAtEndOfBatch() throws Exception { + File f = new File("target", "FastRollingFileAppenderTest.log"); + // System.out.println(f.getAbsolutePath()); + f.delete(); + Logger log = LogManager.getLogger("com.foo.Bar"); + String msg = "Message flushed with immediate flush=false"; + log.info(msg); + ((LifeCycle) LogManager.getContext()).stop(); // stop async thread + + BufferedReader reader = new BufferedReader(new FileReader(f)); + String line1 = reader.readLine(); + reader.close(); + f.delete(); + assertNotNull("line1", line1); + assertTrue("line1 correct", line1.contains(msg)); + + String location = "testFlushAtEndOfBatch"; + assertTrue("no location", !line1.contains(location)); + } +} Index: core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerConfigTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerConfigTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerConfigTest.java (working copy) @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import static org.junit.Assert.*; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LifeCycle; +import org.apache.logging.log4j.core.config.XMLConfigurationFactory; +import org.junit.BeforeClass; +import org.junit.Test; + +public class AsyncLoggerConfigTest { + + @BeforeClass + public static void beforeClass() { + System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, + "AsyncLoggerConfigTest.xml"); + } + + @Test + public void testAdditivity() throws Exception { + File f = new File("target", "AsyncLoggerConfigTest.log"); + // System.out.println(f.getAbsolutePath()); + f.delete(); + Logger log = LogManager.getLogger("com.foo.Bar"); + String msg = "Additive logging: 2 for the price of 1!"; + log.info(msg); + ((LifeCycle) LogManager.getContext()).stop(); // stop async thread + + BufferedReader reader = new BufferedReader(new FileReader(f)); + String line1 = reader.readLine(); + String line2 = reader.readLine(); + reader.close(); + f.delete(); + assertNotNull("line1", line1); + assertNotNull("line2", line2); + assertTrue("line1 correct", line1.contains(msg)); + assertTrue("line2 correct", line2.contains(msg)); + + String location = "testAdditivity"; + assertTrue("location", + line1.contains(location) || line2.contains(location)); + } + +} Index: core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerContextSelectorTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerContextSelectorTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerContextSelectorTest.java (working copy) @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import static org.junit.Assert.*; + +import java.util.List; + +import org.apache.logging.log4j.core.LoggerContext; +import org.apache.logging.log4j.core.async.AsyncLoggerContext; +import org.apache.logging.log4j.core.async.AsyncLoggerContextSelector; +import org.junit.Test; + +public class AsyncLoggerContextSelectorTest { + + @Test + public void testContextReturnsAsyncLoggerContext() { + AsyncLoggerContextSelector selector = new AsyncLoggerContextSelector(); + LoggerContext context = selector.getContext(null, null, false); + + assertTrue(context instanceof AsyncLoggerContext); + } + + @Test + public void testContext2ReturnsAsyncLoggerContext() { + AsyncLoggerContextSelector selector = new AsyncLoggerContextSelector(); + LoggerContext context = selector.getContext(null, null, false, null); + + assertTrue(context instanceof AsyncLoggerContext); + } + + @Test + public void testLoggerContextsReturnsAsyncLoggerContext() { + AsyncLoggerContextSelector selector = new AsyncLoggerContextSelector(); + List list = selector.getLoggerContexts(); + + assertEquals(1, list.size()); + assertTrue(list.get(0) instanceof AsyncLoggerContext); + } + + @Test + public void testContextNameIsAsyncLoggerContext() { + AsyncLoggerContextSelector selector = new AsyncLoggerContextSelector(); + LoggerContext context = selector.getContext(null, null, false); + + assertEquals("AsyncLoggerContext", context.getName()); + } + +} Index: core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerContextTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerContextTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerContextTest.java (working copy) @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import static org.junit.Assert.*; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LifeCycle; +import org.apache.logging.log4j.core.LoggerContext; +import org.apache.logging.log4j.core.async.AsyncLogger; +import org.apache.logging.log4j.core.async.AsyncLoggerContext; +import org.junit.Test; + +public class AsyncLoggerContextTest { + + @Test + public void testNewInstanceReturnsAsyncLogger() { + Logger logger = new AsyncLoggerContext("a").newInstance( + new LoggerContext("a"), "a", null); + assertTrue(logger instanceof AsyncLogger); + + ((LifeCycle) LogManager.getContext()).stop(); // stop async thread + } +} Index: core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerLocationTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerLocationTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerLocationTest.java (working copy) @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import static org.junit.Assert.*; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LifeCycle; +import org.apache.logging.log4j.core.async.AsyncLoggerContextSelector; +import org.apache.logging.log4j.core.config.XMLConfigurationFactory; +import org.apache.logging.log4j.core.helpers.Constants; +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.Test; + +public class AsyncLoggerLocationTest { + + @BeforeClass + public static void beforeClass() { + System.setProperty(Constants.LOG4J_CONTEXT_SELECTOR, + AsyncLoggerContextSelector.class.getName()); + System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, + "AsyncLoggerLocationTest.xml"); + } + + @AfterClass + public static void afterClass() { + System.setProperty(Constants.LOG4J_CONTEXT_SELECTOR, ""); + } + + @Test + public void testAsyncLogWritesToLog() throws Exception { + File f = new File("target", "AsyncLoggerLocationTest.log"); + // System.out.println(f.getAbsolutePath()); + f.delete(); + Logger log = LogManager.getLogger("com.foo.Bar"); + String msg = "Async logger msg with location"; + log.info(msg); + ((LifeCycle) LogManager.getContext()).stop(); // stop async thread + + BufferedReader reader = new BufferedReader(new FileReader(f)); + String line1 = reader.readLine(); + reader.close(); + f.delete(); + assertNotNull("line1", line1); + assertTrue("line1 correct", line1.contains(msg)); + + String location = "testAsyncLogWritesToLog"; + assertTrue("has location", line1.contains(location)); + } + +} Index: core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/AsyncLoggerTest.java (working copy) @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import static org.junit.Assert.*; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LifeCycle; +import org.apache.logging.log4j.core.async.AsyncLoggerContextSelector; +import org.apache.logging.log4j.core.config.XMLConfigurationFactory; +import org.apache.logging.log4j.core.helpers.Constants; +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.Test; + +public class AsyncLoggerTest { + + @BeforeClass + public static void beforeClass() { + System.setProperty(Constants.LOG4J_CONTEXT_SELECTOR, + AsyncLoggerContextSelector.class.getName()); + System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, + "AsyncLoggerTest.xml"); + } + + @AfterClass + public static void afterClass() { + System.setProperty(Constants.LOG4J_CONTEXT_SELECTOR, ""); + } + + @Test + public void testAsyncLogWritesToLog() throws Exception { + File f = new File("target", "AsyncLoggerTest.log"); + // System.out.println(f.getAbsolutePath()); + f.delete(); + Logger log = LogManager.getLogger("com.foo.Bar"); + String msg = "Async logger msg"; + log.info(msg); + ((LifeCycle) LogManager.getContext()).stop(); // stop async thread + + BufferedReader reader = new BufferedReader(new FileReader(f)); + String line1 = reader.readLine(); + reader.close(); + f.delete(); + assertNotNull("line1", line1); + assertTrue("line1 correct", line1.contains(msg)); + + String location = "testAsyncLogWritesToLog"; + assertTrue("no location", !line1.contains(location)); + } + +} Index: core/src/test/java/org/apache/logging/log4j/core/async/CachedClockTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/CachedClockTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/CachedClockTest.java (working copy) @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import static org.junit.Assert.*; + +import org.apache.logging.log4j.core.async.CachedClock; +import org.junit.Test; + +public class CachedClockTest { + + @Test + public void testLessThan17Millis() { + long millis1 = CachedClock.instance().currentTimeMillis(); + long sysMillis = System.currentTimeMillis(); + + long diff = sysMillis - millis1; + + assertTrue("diff too large: " + diff, diff <= 16); + } + + @Test + public void testAfterWaitStillLessThan17Millis() throws Exception { + Thread.sleep(100); + long millis1 = CachedClock.instance().currentTimeMillis(); + long sysMillis = System.currentTimeMillis(); + + long diff = sysMillis - millis1; + + assertTrue("diff too large: " + diff, diff <= 16); + } + +} Index: core/src/test/java/org/apache/logging/log4j/core/async/ClockFactoryTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/ClockFactoryTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/ClockFactoryTest.java (working copy) @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import static org.junit.Assert.*; + +import org.apache.logging.log4j.core.async.CachedClock; +import org.apache.logging.log4j.core.async.Clock; +import org.apache.logging.log4j.core.async.ClockFactory; +import org.apache.logging.log4j.core.async.CoarseCachedClock; +import org.apache.logging.log4j.core.async.SystemClock; +import org.junit.Test; + +public class ClockFactoryTest { + + @Test + public void testDefaultIsSystemClock() { + System.clearProperty(ClockFactory.PROPERTY_NAME); + assertEquals(SystemClock.class, ClockFactory.getClock().getClass()); + } + + @Test + public void testSpecifySystemClockShort() { + System.setProperty(ClockFactory.PROPERTY_NAME, "SystemClock"); + assertEquals(SystemClock.class, ClockFactory.getClock().getClass()); + } + + @Test + public void testSpecifySystemClockLong() { + System.setProperty(ClockFactory.PROPERTY_NAME, SystemClock.class.getName()); + assertEquals(SystemClock.class, ClockFactory.getClock().getClass()); + } + + @Test + public void testSpecifyCachedClockShort() { + System.setProperty(ClockFactory.PROPERTY_NAME, "CachedClock"); + assertEquals(CachedClock.class, ClockFactory.getClock().getClass()); + } + + @Test + public void testSpecifyCachedClockLong() { + System.setProperty(ClockFactory.PROPERTY_NAME, CachedClock.class.getName()); + assertEquals(CachedClock.class, ClockFactory.getClock().getClass()); + } + + @Test + public void testSpecifyCoarseCachedClockShort() { + System.setProperty(ClockFactory.PROPERTY_NAME, "CoarseCachedClock"); + assertEquals(CoarseCachedClock.class, ClockFactory.getClock().getClass()); + } + + @Test + public void testSpecifyCoarseCachedClockLong() { + System.setProperty(ClockFactory.PROPERTY_NAME, CoarseCachedClock.class.getName()); + assertEquals(CoarseCachedClock.class, ClockFactory.getClock().getClass()); + } + + static class MyClock implements Clock { + @Override + public long currentTimeMillis() { + return 42; + } + } + + @Test + public void testCustomClock() { + System.setProperty(ClockFactory.PROPERTY_NAME, MyClock.class.getName()); + assertEquals(MyClock.class, ClockFactory.getClock().getClass()); + } + +} Index: core/src/test/java/org/apache/logging/log4j/core/async/SystemClockTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/SystemClockTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/SystemClockTest.java (working copy) @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache license, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the license for the specific language governing permissions and + * limitations under the license. + */ +package org.apache.logging.log4j.core.async; + +import static org.junit.Assert.*; + +import org.apache.logging.log4j.core.async.SystemClock; +import org.junit.Test; + +public class SystemClockTest { + + @Test + public void testLessThan2Millis() { + long millis1 = new SystemClock().currentTimeMillis(); + long sysMillis = System.currentTimeMillis(); + + long diff = sysMillis - millis1; + + assertTrue("diff too large: " + diff, diff <= 1); + } + + @Test + public void testAfterWaitStillLessThan2Millis() throws Exception { + Thread.sleep(100); + long millis1 = new SystemClock().currentTimeMillis(); + long sysMillis = System.currentTimeMillis(); + + long diff = sysMillis - millis1; + + assertTrue("diff too large: " + diff, diff <= 1); + } + +} Index: core/src/test/java/org/apache/logging/log4j/core/async/perftest/IPerfTestRunner.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/perftest/IPerfTestRunner.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/perftest/IPerfTestRunner.java (working copy) @@ -0,0 +1,17 @@ +package org.apache.logging.log4j.core.async.perftest; + +import com.lmax.disruptor.collections.Histogram; + +public interface IPerfTestRunner { + static final String LINE100 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890!\"#$%&'()-=^~|\\@`[]{};:+*,.<>/?_123456"; + static final String THROUGHPUT_MSG = LINE100 + LINE100 + LINE100 + LINE100 + + LINE100; + static final String LATENCY_MSG = "Short msg"; + + void runThroughputTest(int lines, Histogram histogram); + + void runLatencyTest(int samples, Histogram histogram, long nanoTimeCost, + int threadCount); + void shutdown(); + void log(String finalMessage); +} Index: core/src/test/java/org/apache/logging/log4j/core/async/perftest/MTPerfTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/perftest/MTPerfTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/perftest/MTPerfTest.java (working copy) @@ -0,0 +1,93 @@ +package org.apache.logging.log4j.core.async.perftest; + +import java.io.File; + +import com.lmax.disruptor.collections.Histogram; + +public class MTPerfTest extends PerfTest { + + public static void main(String[] args) throws Exception { + new MTPerfTest().doMain(args); + } + + @Override + public void runTestAndPrintResult(final IPerfTestRunner runner, + final String name, final int threadCount, String resultFile) + throws Exception { + + // ThreadContext.put("aKey", "mdcVal"); + PerfTest.println("Warming up the JVM..."); + long t1 = System.nanoTime(); + + // warmup at least 2 rounds and at most 1 minute + final Histogram warmupHist = PerfTest.createHistogram(); + final long stop = System.currentTimeMillis() + (60 * 1000); + Runnable run1 = new Runnable() { + public void run() { + for (int i = 0; i < 10; i++) { + final int LINES = PerfTest.throughput ? 50000 : 200000; + runTest(runner, LINES, null, warmupHist, 2); + if (i > 0 && System.currentTimeMillis() >= stop) { + return; + } + } + } + }; + Thread thread1 = new Thread(run1); + Thread thread2 = new Thread(run1); + thread1.start(); + thread2.start(); + thread1.join(); + thread2.join(); + + PerfTest.printf("Warmup complete in %.1f seconds%n", + (System.nanoTime() - t1) / (1000.0 * 1000.0 * 1000.0)); + PerfTest.println("Waiting 10 seconds for buffers to drain warmup data..."); + Thread.sleep(10000); + new File("perftest.log").delete(); + new File("perftest.log").createNewFile(); + + PerfTest.println("Starting the main test..."); + PerfTest.throughput = false; + multiThreadedTestRun(runner, name, threadCount, resultFile); + + Thread.sleep(1000); + PerfTest.throughput = true; + multiThreadedTestRun(runner, name, threadCount, resultFile); + } + + private void multiThreadedTestRun(final IPerfTestRunner runner, + final String name, final int threadCount, String resultFile) + throws Exception { + + final Histogram[] histograms = new Histogram[threadCount]; + for (int i = 0; i < histograms.length; i++) { + histograms[i] = PerfTest.createHistogram(); + } + final int LINES = 256 * 1024; + + Thread[] threads = new Thread[threadCount]; + for (int i = 0; i < threads.length; i++) { + final Histogram histogram = histograms[i]; + threads[i] = new Thread() { + public void run() { +// int latencyCount = threadCount >= 16 ? 1000000 : 5000000; + int latencyCount = 5000000; + int count = PerfTest.throughput ? LINES / threadCount + : latencyCount; + runTest(runner, count, "end", histogram, threadCount); + } + }; + } + for (Thread thread : threads) { + thread.start(); + } + for (Thread thread : threads) { + thread.join(); + } + + for (Histogram histogram : histograms) { + PerfTest.reportResult(resultFile, name, histogram); + } + } +} \ No newline at end of file Index: core/src/test/java/org/apache/logging/log4j/core/async/perftest/PerfTest.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/perftest/PerfTest.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/perftest/PerfTest.java (working copy) @@ -0,0 +1,168 @@ +package org.apache.logging.log4j.core.async.perftest; + +import java.io.FileWriter; +import java.io.IOException; + +import com.lmax.disruptor.collections.Histogram; + +public class PerfTest { + + private static final String LINE100 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890!\"#$%&'()-=^~|\\@`[]{};:+*,.<>/?_123456"; + public static final String LINE500 = LINE100 + LINE100 + LINE100 + LINE100 + + LINE100; + + static boolean verbose = false; + static boolean throughput; + + // determine how long it takes to call System.nanoTime() (on average) + static long calcNanoTimeCost() { + final long iterations = 10000000; + long start = System.nanoTime(); + long finish = start; + + for (int i = 0; i < iterations; i++) { + finish = System.nanoTime(); + } + + if (finish <= start) { + throw new IllegalStateException(); + } + + finish = System.nanoTime(); + return (finish - start) / iterations; + } + + static Histogram createHistogram() { + long[] intervals = new long[31]; + long intervalUpperBound = 1L; + for (int i = 0, size = intervals.length - 1; i < size; i++) { + intervalUpperBound *= 2; + intervals[i] = intervalUpperBound; + } + + intervals[intervals.length - 1] = Long.MAX_VALUE; + return new Histogram(intervals); + } + + public static void main(String[] args) throws Exception { + new PerfTest().doMain(args); + } + + public void doMain(String[] args) throws Exception { + String runnerClass = args[0]; + IPerfTestRunner runner = (IPerfTestRunner) Class.forName(runnerClass) + .newInstance(); + String name = args[1]; + String resultFile = args.length > 2 ? args[2] : null; + for (String arg : args) { + if ("-verbose".equalsIgnoreCase(arg)) { + verbose = true; + } + if ("-throughput".equalsIgnoreCase(arg)) { + throughput = true; + } + } + int threadCount = args.length > 2 ? Integer.parseInt(args[3]) : 3; + printf("Starting %s %s (%d)...%n", getClass().getSimpleName(), name, + threadCount); + runTestAndPrintResult(runner, name, threadCount, resultFile); + runner.shutdown(); + System.exit(0); + } + + public void runTestAndPrintResult(IPerfTestRunner runner, + final String name, int threadCount, String resultFile) + throws Exception { + Histogram warmupHist = createHistogram(); + + // ThreadContext.put("aKey", "mdcVal"); + println("Warming up the JVM..."); + long t1 = System.nanoTime(); + + // warmup at least 2 rounds and at most 1 minute + final long stop = System.currentTimeMillis() + (60 * 1000); + for (int i = 0; i < 10; i++) { + final int LINES = throughput ? 50000 : 200000; + runTest(runner, LINES, null, warmupHist, 1); + if (i > 0 && System.currentTimeMillis() >= stop) { + return; + } + } + + printf("Warmup complete in %.1f seconds%n", (System.nanoTime() - t1) + / (1000.0 * 1000.0 * 1000.0)); + println("Waiting 10 seconds for buffers to drain warmup data..."); + Thread.sleep(10000); + + println("Starting the main test..."); + // test + throughput = false; + runSingleThreadedTest(runner, name, resultFile); + + Thread.sleep(1000); + + throughput = true; + runSingleThreadedTest(runner, name, resultFile); + } + + private int runSingleThreadedTest(IPerfTestRunner runner, String name, + String resultFile) throws IOException { + Histogram latency = createHistogram(); + final int LINES = throughput ? 50000 : 5000000; + runTest(runner, LINES, "end", latency, 1); + reportResult(resultFile, name, latency); + return LINES; + } + + static void reportResult(String file, String name, Histogram histogram) + throws IOException { + String result = createSamplingReport(name, histogram); + println(result); + + if (file != null) { + FileWriter writer = new FileWriter(file, true); + writer.write(result); + writer.write(System.getProperty("line.separator")); + writer.close(); + } + } + + static void printf(String msg, Object... objects) { + if (verbose) { + System.out.printf(msg, objects); + } + } + + static void println(String msg) { + if (verbose) { + System.out.println(msg); + } + } + + static String createSamplingReport(String name, Histogram histogram) { + Histogram data = histogram; + if (throughput) { + return data.getMax() + " operations/second"; + } + String result = String.format( + "avg=%.0f 99%%=%d 99.99%%=%d sampleCount=%d", data.getMean(), // + data.getTwoNinesUpperBound(), // + data.getFourNinesUpperBound(), // + data.getCount() // + ); + return result; + } + + public void runTest(IPerfTestRunner runner, int lines, String finalMessage, + Histogram histogram, int threadCount) { + if (throughput) { + runner.runThroughputTest(lines, histogram); + } else { + long nanoTimeCost = calcNanoTimeCost(); + runner.runLatencyTest(lines, histogram, nanoTimeCost, threadCount); + } + if (finalMessage != null) { + runner.log(finalMessage); + } + } +} Index: core/src/test/java/org/apache/logging/log4j/core/async/perftest/PerfTestDriver.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/perftest/PerfTestDriver.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/perftest/PerfTestDriver.java (working copy) @@ -0,0 +1,359 @@ +package org.apache.logging.log4j.core.async.perftest; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; +import java.io.IOException; +import java.io.InputStreamReader; +import java.nio.CharBuffer; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import org.apache.logging.log4j.core.async.AsyncLoggerContextSelector; + +/** + * Runs a sequence of performance tests. + */ +public class PerfTestDriver { + static enum WaitStrategy { + Sleep, Yield, Block + }; + + /** + * Defines the setup for a java process running a performance test. + */ + static class Setup implements Comparable { + private Class _class; + private String _log4jConfig; + private String _name; + private String[] _systemProperties; + private int _threadCount; + private File _temp; + public Stats _stats; + private WaitStrategy _wait; + private String _runner; + + public Setup(Class klass, String runner, String name, + String log4jConfig, int threadCount, WaitStrategy wait, + String... systemProperties) throws IOException { + _class = klass; + _runner = runner; + _name = name; + _log4jConfig = log4jConfig; + _threadCount = threadCount; + _systemProperties = systemProperties; + _wait = wait; + _temp = File.createTempFile("log4jperformance", ".txt"); + } + + List processArguments(String java) { + List args = new ArrayList(); + args.add(java); + args.add("-server"); + args.add("-Xms1g"); + args.add("-Xmx1g"); + + // args.add("-XX:+UseParallelOldGC"); + // args.add("-Xloggc:gc.log"); + // args.add("-XX:+PrintGCTimeStamps"); + // args.add("-XX:+PrintGCDetails"); + // args.add("-XX:+PrintGCDateStamps"); + // args.add("-XX:+PrintGCApplicationStoppedTime"); + // args.add("-XX:+PrintGCApplicationConcurrentTime"); + // args.add("-XX:+PrintSafepointStatistics"); + + args.add("-Dlog4j.configuration=" + _log4jConfig); // log4j1.2 + args.add("-Dlog4j.configurationFile=" + _log4jConfig); // log4j2 + args.add("-Dlogback.configurationFile=" + _log4jConfig);// logback + + int ringBufferSize = getUserSpecifiedRingBufferSize(); + if (ringBufferSize >= 128) { + args.add("-DAsyncLoggerConfig.RingBufferSize=" + ringBufferSize); + args.add("-DAsyncLogger.RingBufferSize=" + ringBufferSize); + } + args.add("-DAsyncLoggerConfig.WaitStrategy=" + _wait); + args.add("-DAsyncLogger.WaitStrategy=" + _wait); + if (_systemProperties != null) { + for (String property : _systemProperties) { + args.add(property); + } + } + args.add("-cp"); + args.add(System.getProperty("java.class.path")); + args.add(_class.getName()); + args.add(_runner); + args.add(_name); + args.add(_temp.getAbsolutePath()); + args.add(String.valueOf(_threadCount)); + return args; + } + + private int getUserSpecifiedRingBufferSize() { + try { + return Integer.parseInt(System.getProperty("RingBufferSize", + "-1")); + } catch (Exception ignored) { + return -1; + } + } + + ProcessBuilder latencyTest(String java) { + return new ProcessBuilder(processArguments(java)); + } + + ProcessBuilder throughputTest(String java) { + List args = processArguments(java); + args.add("-throughput"); + return new ProcessBuilder(args); + } + + @Override + public int compareTo(Setup other) { + // largest ops/sec first + return (int) Math.signum(other._stats._averageOpsPerSec + - _stats._averageOpsPerSec); + } + + public String description() { + String detail = _class.getSimpleName(); + if (PerfTest.class == _class) { + detail = "single thread"; + } else if (MTPerfTest.class == _class) { + detail = _threadCount + " threads"; + } + String target = _runner.substring(_runner.indexOf(".Run") + 4); + return target + ": " + _name + " (" + detail + ")"; + } + } + + /** + * Results of a performance test. + */ + static class Stats { + int _count; + long _average; + long _pct99; + long _pct99_99; + double _latencyRowCount; + int _throughputRowCount; + private long _averageOpsPerSec; + + // example line: avg=828 99%=1118 99.99%=5028 Count=3125 + public Stats(String raw, int repeat) { + String[] lines = raw.split("[\\r\\n]+"); + long totalOps = 0; + for (String line : lines) { + if (line.startsWith("avg")) { + _latencyRowCount++; + String[] parts = line.split(" "); + int i = 0; + _average += Long.parseLong(parts[i++].split("=")[1]); + _pct99 += Long.parseLong(parts[i++].split("=")[1]); + _pct99_99 += Long.parseLong(parts[i++].split("=")[1]); + _count += Integer.parseInt(parts[i++].split("=")[1]); + } else { + _throughputRowCount++; + String number = line.substring(0, line.indexOf(' ')); + long opsPerSec = Long.parseLong(number); + totalOps += opsPerSec; + } + } + _averageOpsPerSec = totalOps / (int) _throughputRowCount; + } + + public String toString() { + String fmt = "throughput: %,d ops/sec. latency(ns): avg=%.1f 99%% < %.1f 99.99%% < %.1f (%d samples)"; + return String.format(fmt, _averageOpsPerSec, // + _average / _latencyRowCount, // mean latency + _pct99 / _latencyRowCount, // 99% observations less than + _pct99_99 / _latencyRowCount,// 99.99% observs less than + _count); + } + } + + // single-threaded performance test + private static Setup s(String config, String runner, String name, + String... systemProperties) throws IOException { + WaitStrategy wait = WaitStrategy.valueOf(System.getProperty( + "WaitStrategy", "Sleep")); + return new Setup(PerfTest.class, runner, name, config, 1, wait, + systemProperties); + } + + // multi-threaded performance test + private static Setup m(String config, String runner, String name, + int threadCount, String... systemProperties) throws IOException { + WaitStrategy wait = WaitStrategy.valueOf(System.getProperty( + "WaitStrategy", "Sleep")); + return new Setup(MTPerfTest.class, runner, name, config, threadCount, + wait, systemProperties); + } + + public static void main(String[] args) throws Exception { + final String ALL_ASYNC = "-DLog4jContextSelector=" + + AsyncLoggerContextSelector.class.getName(); + final String CACHEDCLOCK = "-DAsyncLogger.Clock=CachedClock"; + final String SYSCLOCK = ""; + final String LOG12 = RunLog4j1.class.getName(); + final String LOG20 = RunLog4j2.class.getName(); + final String LOGBK = RunLogback.class.getName(); + + long start = System.nanoTime(); + List tests = new ArrayList(); + // includeLocation=false + tests.add(s("perf3PlainNoLoc.xml", LOG20, "Loggers all async", + ALL_ASYNC, SYSCLOCK)); + tests.add(s("perf7MixedNoLoc.xml", LOG20, "Loggers mixed sync/async")); + tests.add(s("perf-logback.xml", LOGBK, "Sync")); + tests.add(s("perf-log4j12.xml", LOG12, "Sync")); + tests.add(s("perf3PlainNoLoc.xml", LOG20, "Sync")); + tests.add(s("perf-logback-async.xml", LOGBK, "Async Appender")); + tests.add(s("perf-log4j12-async.xml", LOG12, "Async Appender")); + tests.add(s("perf5AsyncApndNoLoc.xml", LOG20, "Async Appender")); + + // includeLocation=true + // tests.add(s("perf6AsyncApndLoc.xml", LOG20, + // "Async Appender includeLocation")); + // tests.add(s("perf8MixedLoc.xml", LOG20, + // "Mixed sync/async includeLocation")); + // tests.add(s("perf4PlainLocation.xml", LOG20, + // "Loggers all async includeLocation", ALL_ASYNC)); + // tests.add(s("perf4PlainLocation.xml", LOG20, + // "Loggers all async includeLocation CachedClock", ALL_ASYNC, + // CACHEDCLOCK)); + // tests.add(s("perf4PlainLocation.xml", LOG20, + // "Sync includeLocation")); + + // appenders + // tests.add(s("perf1syncFile.xml", LOG20, "FileAppender")); + // tests.add(s("perf1syncFastFile.xml", LOG20, "FastFileAppender")); + // tests.add(s("perf2syncRollFile.xml", LOG20, "RollFileAppender")); + // tests.add(s("perf2syncRollFastFile.xml", LOG20, + // "RollFastFileAppender")); + + final int MAX_THREADS = 16; // 64 takes a LONG time + for (int i = 2; i <= MAX_THREADS; i *= 2) { + // includeLocation = false + tests.add(m("perf-logback.xml", LOGBK, "Sync", i)); + tests.add(m("perf-log4j12.xml", LOG12, "Sync", i)); + tests.add(m("perf3PlainNoLoc.xml", LOG20, "Sync", i)); + tests.add(m("perf-logback-async.xml", LOGBK, "Async Appender", i)); + tests.add(m("perf-log4j12-async.xml", LOG12, "Async Appender", i)); + tests.add(m("perf5AsyncApndNoLoc.xml", LOG20, "Async Appender", i)); + tests.add(m("perf3PlainNoLoc.xml", LOG20, "Loggers all async", i, + ALL_ASYNC, SYSCLOCK)); + tests.add(m("perf7MixedNoLoc.xml", LOG20, + "Loggers mixed sync/async", i)); + + // includeLocation=true + // tests.add(m("perf6AsyncApndLoc.xml", LOG20, + // "Async Appender includeLocation", i)); + // tests.add(m("perf8MixedLoc.xml", LOG20, + // "Mixed sync/async includeLocation", i)); + // tests.add(m("perf4PlainLocation.xml", LOG20, + // "Loggers all async includeLocation", i, ALL_ASYNC)); + // tests.add(m("perf4PlainLocation.xml", LOG20, + // "Loggers all async includeLocation CachedClock", i, + // ALL_ASYNC, CACHEDCLOCK)); + // tests.add(m("perf4PlainLocation.xml", LOG20, + // "Sync includeLocation", i)); + + // appenders + // tests.add(m("perf1syncFile.xml", LOG20, "FileAppender", i)); + // tests.add(m("perf1syncFastFile.xml", LOG20, "FastFileAppender", + // i)); + // tests.add(m("perf2syncRollFile.xml", LOG20, "RollFileAppender", + // i)); + // tests.add(m("perf2syncRollFastFile.xml", LOG20, + // "RollFastFileAppender", i)); + } + + String java = args.length > 0 ? args[0] : "java"; + int repeat = args.length > 1 ? Integer.parseInt(args[1]) : 5; + int x = 0; + for (Setup config : tests) { + System.out.print(config.description()); + ProcessBuilder pb = config.throughputTest(java); + pb.redirectErrorStream(true); // merge System.out and System.err + long t1 = System.nanoTime(); + // int count = config._threadCount >= 16 ? 2 : repeat; + int count = repeat; + runPerfTest(count, x++, config, pb); + System.out.printf(" took %.1f seconds%n", (System.nanoTime() - t1) + / (1000.0 * 1000.0 * 1000.0)); + + FileReader reader = new FileReader(config._temp); + CharBuffer buffer = CharBuffer.allocate(256 * 1024); + reader.read(buffer); + reader.close(); + config._temp.delete(); + buffer.flip(); + + String raw = buffer.toString(); + System.out.print(raw); + Stats stats = new Stats(raw, repeat); + System.out.println(stats); + System.out.println("-----"); + config._stats = stats; + } + new File("perftest.log").delete(); + System.out + .printf("Done. Total duration: %.1f minutes%n", + (System.nanoTime() - start) + / (60.0 * 1000.0 * 1000.0 * 1000.0)); + + printRanking((Setup[]) tests.toArray(new Setup[tests.size()])); + } + + private static void printRanking(Setup[] tests) { + System.out.println(); + System.out.println("Ranking:"); + Arrays.sort(tests); + for (int i = 0; i < tests.length; i++) { + Setup setup = tests[i]; + System.out.println((i + 1) + ". " + setup.description() + ": " + + setup._stats); + } + } + + private static void runPerfTest(int repeat, int setupIndex, Setup config, + ProcessBuilder pb) throws IOException, InterruptedException { + for (int i = 0; i < repeat; i++) { + System.out.print(" (" + (i + 1) + "/" + repeat + ")..."); + final Process process = pb.start(); + + final boolean[] stop = { false }; + printProcessOutput(process, stop); + process.waitFor(); + stop[0] = true; + + File gc = new File("gc" + setupIndex + "_" + i + + config._log4jConfig + ".log"); + if (gc.exists()) { + gc.delete(); + } + new File("gc.log").renameTo(gc); + } + } + + private static Thread printProcessOutput(final Process process, + final boolean[] stop) { + + Thread t = new Thread("OutputWriter") { + public void run() { + BufferedReader in = new BufferedReader(new InputStreamReader( + process.getInputStream())); + try { + String line = null; + while (!stop[0] && (line = in.readLine()) != null) { + System.out.println(line); + } + } catch (Exception ignored) { + } + } + }; + t.start(); + return t; + } +} Index: core/src/test/java/org/apache/logging/log4j/core/async/perftest/PerfTestResultFormatter.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/perftest/PerfTestResultFormatter.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/perftest/PerfTestResultFormatter.java (working copy) @@ -0,0 +1,167 @@ +package org.apache.logging.log4j.core.async.perftest; + +import java.io.BufferedReader; +import java.io.InputStreamReader; +import java.text.DecimalFormat; +import java.text.NumberFormat; +import java.text.ParseException; +import java.util.Arrays; +import java.util.Comparator; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TreeMap; + +/** + * Utility class that can read the "Ranking" output of the PerfTestDriver and + * format it for pasting into Excel. + */ +class PerfTestResultFormatter { + static final String LF = System.getProperty("line.separator"); + static final NumberFormat NUM = new DecimalFormat("#,##0"); + + static class Stats { + long throughput; + double avgLatency; + double latency99Pct; + double latency99_99Pct; + + Stats(String throughput, String avg, String lat99, String lat99_99) + throws ParseException { + this.throughput = NUM.parse(throughput.trim()).longValue(); + this.avgLatency = Double.parseDouble(avg.trim()); + this.latency99Pct = Double.parseDouble(lat99.trim()); + this.latency99_99Pct = Double.parseDouble(lat99_99.trim()); + } + } + + private Map> results = new TreeMap>(); + + public PerfTestResultFormatter() { + } + + public String format(String text) throws ParseException { + results.clear(); + String[] lines = text.split("[\\r\\n]+"); + for (String line : lines) { + process(line); + } + return latencyTable() + LF + throughputTable(); + } + + private String latencyTable() { + StringBuilder sb = new StringBuilder(4 * 1024); + Set subKeys = results.values().iterator().next().keySet(); + char[] tabs = new char[subKeys.size()]; + Arrays.fill(tabs, '\t'); + String sep = new String(tabs); + sb.append("\tAverage latency" + sep + "99% less than" + sep + + "99.99% less than"); + sb.append(LF); + for (int i = 0; i < 3; i++) { + for (String subKey : subKeys) { + sb.append("\t").append(subKey); + } + } + sb.append(LF); + for (String key : results.keySet()) { + sb.append(key); + for (int i = 0; i < 3; i++) { + Map sub = results.get(key); + for (String subKey : sub.keySet()) { + Stats stats = sub.get(subKey); + switch (i) { + case 0: + sb.append("\t").append((long) stats.avgLatency); + break; + case 1: + sb.append("\t").append((long) stats.latency99Pct); + break; + case 2: + sb.append("\t").append((long) stats.latency99_99Pct); + break; + } + } + } + sb.append(LF); + } + return sb.toString(); + } + + private String throughputTable() { + StringBuilder sb = new StringBuilder(4 * 1024); + Set subKeys = results.values().iterator().next().keySet(); + sb.append("\tThroughput per thread (msg/sec)"); + sb.append(LF); + for (String subKey : subKeys) { + sb.append("\t").append(subKey); + } + sb.append(LF); + for (String key : results.keySet()) { + sb.append(key); + Map sub = results.get(key); + for (String subKey : sub.keySet()) { + Stats stats = sub.get(subKey); + sb.append("\t").append((long) stats.throughput); + } + sb.append(LF); + } + return sb.toString(); + } + + private void process(String line) throws ParseException { + String key = line.substring(line.indexOf('.') + 1, line.indexOf('(')); + String sub = line.substring(line.indexOf('(') + 1, line.indexOf(')')); + String throughput = line.substring(line.indexOf("throughput: ") + + "throughput: ".length(), line.indexOf(" ops")); + String avg = line.substring(line.indexOf("avg=") + "avg=".length(), + line.indexOf(" 99%")); + String pct99 = line.substring( + line.indexOf("99% < ") + "99% < ".length(), + line.indexOf(" 99.99%")); + String pct99_99 = line.substring(line.indexOf("99.99% < ") + + "99.99% < ".length(), line.lastIndexOf('(') - 1); + Stats stats = new Stats(throughput, avg, pct99, pct99_99); + Map map = results.get(key.trim()); + if (map == null) { + map = new TreeMap(sort()); + results.put(key.trim(), map); + } + String subKey = sub.trim(); + if ("single thread".equals(subKey)) { + subKey = "1 thread"; + } + map.put(subKey, stats); + } + + private Comparator sort() { + return new Comparator() { + List expected = Arrays.asList("1 thread", "2 threads", + "4 threads", "8 threads", "16 threads", "32 threads", + "64 threads"); + + @Override + public int compare(String o1, String o2) { + int i1 = expected.indexOf(o1); + int i2 = expected.indexOf(o2); + if (i1 < 0 || i2 < 0) { + return o1.compareTo(o2); + } + return i1 - i2; + } + }; + } + + public static void main(String[] args) throws Exception { + PerfTestResultFormatter fmt = new PerfTestResultFormatter(); + BufferedReader reader = new BufferedReader(new InputStreamReader( + System.in)); + String line = null; + while ((line = reader.readLine()) != null) { + fmt.process(line); + } + System.out.println(fmt.latencyTable()); + System.out.println(); + System.out.println(fmt.throughputTable()); + } +} \ No newline at end of file Index: core/src/test/java/org/apache/logging/log4j/core/async/perftest/RunLog4j1.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/perftest/RunLog4j1.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/perftest/RunLog4j1.java (working copy) @@ -0,0 +1,53 @@ +package org.apache.logging.log4j.core.async.perftest; + +import org.apache.log4j.LogManager; +import org.apache.log4j.Logger; + +import com.lmax.disruptor.collections.Histogram; + +public class RunLog4j1 implements IPerfTestRunner { + + @Override + public void runThroughputTest(int lines, Histogram histogram) { + long s1 = System.nanoTime(); + Logger logger = LogManager.getLogger(getClass()); + for (int j = 0; j < lines; j++) { + logger.info(THROUGHPUT_MSG); + } + long s2 = System.nanoTime(); + long opsPerSec = (1000L * 1000L * 1000L * lines) / (s2 - s1); + histogram.addObservation(opsPerSec); + } + + @Override + public void runLatencyTest(int samples, Histogram histogram, + long nanoTimeCost, int threadCount) { + Logger logger = LogManager.getLogger(getClass()); + for (int i = 0; i < samples; i++) { + long s1 = System.nanoTime(); + logger.info(LATENCY_MSG); + long s2 = System.nanoTime(); + long value = s2 - s1 - nanoTimeCost; + if (value > 0) { + histogram.addObservation(value); + } + // wait 1 microsec + final long PAUSE_NANOS = 10000 * threadCount; + long pauseStart = System.nanoTime(); + while (PAUSE_NANOS > (System.nanoTime() - pauseStart)) { + // busy spin + } + } + } + + @Override + public void shutdown() { + LogManager.shutdown(); + } + + @Override + public void log(String finalMessage) { + Logger logger = LogManager.getLogger(getClass()); + logger.info(finalMessage); + } +} Index: core/src/test/java/org/apache/logging/log4j/core/async/perftest/RunLog4j2.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/perftest/RunLog4j2.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/perftest/RunLog4j2.java (working copy) @@ -0,0 +1,57 @@ +package org.apache.logging.log4j.core.async.perftest; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.apache.logging.log4j.core.LifeCycle; + +import com.lmax.disruptor.collections.Histogram; + +public class RunLog4j2 implements IPerfTestRunner { + + @Override + public void runThroughputTest(int lines, Histogram histogram) { + long s1 = System.nanoTime(); + Logger logger = LogManager.getLogger(getClass()); + for (int j = 0; j < lines; j++) { + logger.info(THROUGHPUT_MSG); + } + long s2 = System.nanoTime(); + long opsPerSec = (1000L * 1000L * 1000L * lines) / (s2 - s1); + histogram.addObservation(opsPerSec); + } + + + @Override + public void runLatencyTest(int samples, Histogram histogram, + long nanoTimeCost, int threadCount) { + Logger logger = LogManager.getLogger(getClass()); + for (int i = 0; i < samples; i++) { + long s1 = System.nanoTime(); + logger.info(LATENCY_MSG); + long s2 = System.nanoTime(); + long value = s2 - s1 - nanoTimeCost; + if (value > 0) { + histogram.addObservation(value); + } + // wait 1 microsec + final long PAUSE_NANOS = 10000 * threadCount; + long pauseStart = System.nanoTime(); + while (PAUSE_NANOS > (System.nanoTime() - pauseStart)) { + // busy spin + } + } + } + + + @Override + public void shutdown() { + ((LifeCycle) LogManager.getContext()).stop(); // stop async thread + } + + + @Override + public void log(String finalMessage) { + Logger logger = LogManager.getLogger(getClass()); + logger.info(finalMessage); + } +} Index: core/src/test/java/org/apache/logging/log4j/core/async/perftest/RunLogback.java =================================================================== --- core/src/test/java/org/apache/logging/log4j/core/async/perftest/RunLogback.java (revision 0) +++ core/src/test/java/org/apache/logging/log4j/core/async/perftest/RunLogback.java (working copy) @@ -0,0 +1,55 @@ +package org.apache.logging.log4j.core.async.perftest; + +import org.slf4j.LoggerFactory; + +import ch.qos.logback.classic.Logger; +import ch.qos.logback.core.spi.LifeCycle; + +import com.lmax.disruptor.collections.Histogram; + +public class RunLogback implements IPerfTestRunner { + + @Override + public void runThroughputTest(int lines, Histogram histogram) { + long s1 = System.nanoTime(); + Logger logger = (Logger) LoggerFactory.getLogger(getClass()); + for (int j = 0; j < lines; j++) { + logger.info(THROUGHPUT_MSG); + } + long s2 = System.nanoTime(); + long opsPerSec = (1000L * 1000L * 1000L * lines) / (s2 - s1); + histogram.addObservation(opsPerSec); + } + + @Override + public void runLatencyTest(int samples, Histogram histogram, + long nanoTimeCost, int threadCount) { + Logger logger = (Logger) LoggerFactory.getLogger(getClass()); + for (int i = 0; i < samples; i++) { + long s1 = System.nanoTime(); + logger.info(LATENCY_MSG); + long s2 = System.nanoTime(); + long value = s2 - s1 - nanoTimeCost; + if (value > 0) { + histogram.addObservation(value); + } + // wait 1 microsec + final long PAUSE_NANOS = 10000 * threadCount; + long pauseStart = System.nanoTime(); + while (PAUSE_NANOS > (System.nanoTime() - pauseStart)) { + // busy spin + } + } + } + + @Override + public void shutdown() { + ((LifeCycle) LoggerFactory.getILoggerFactory()).stop(); + } + + @Override + public void log(String msg) { + Logger logger = (Logger) LoggerFactory.getLogger(getClass()); + logger.info(msg); + } +} Index: core/src/test/resources/AsyncLoggerConfigTest.xml =================================================================== --- core/src/test/resources/AsyncLoggerConfigTest.xml (revision 0) +++ core/src/test/resources/AsyncLoggerConfigTest.xml (working copy) @@ -0,0 +1,19 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %location %ex%n + + + + + + + + + + + + + \ No newline at end of file Index: core/src/test/resources/AsyncLoggerLocationTest.xml =================================================================== --- core/src/test/resources/AsyncLoggerLocationTest.xml (revision 0) +++ core/src/test/resources/AsyncLoggerLocationTest.xml (working copy) @@ -0,0 +1,17 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %location %m %ex%n + + + + + + + + + + \ No newline at end of file Index: core/src/test/resources/AsyncLoggerTest.xml =================================================================== --- core/src/test/resources/AsyncLoggerTest.xml (revision 0) +++ core/src/test/resources/AsyncLoggerTest.xml (working copy) @@ -0,0 +1,17 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %location %m %ex%n + + + + + + + + + + \ No newline at end of file Index: core/src/test/resources/FastFileAppenderLocationTest.xml =================================================================== --- core/src/test/resources/FastFileAppenderLocationTest.xml (revision 0) +++ core/src/test/resources/FastFileAppenderLocationTest.xml (working copy) @@ -0,0 +1,19 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %location %ex%n + + + + + + + + + + + \ No newline at end of file Index: core/src/test/resources/FastFileAppenderTest.xml =================================================================== --- core/src/test/resources/FastFileAppenderTest.xml (revision 0) +++ core/src/test/resources/FastFileAppenderTest.xml (working copy) @@ -0,0 +1,19 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %location %ex%n + + + + + + + + + + + \ No newline at end of file Index: core/src/test/resources/FastRollingFileAppenderLocationTest.xml =================================================================== --- core/src/test/resources/FastRollingFileAppenderLocationTest.xml (revision 0) +++ core/src/test/resources/FastRollingFileAppenderLocationTest.xml (working copy) @@ -0,0 +1,21 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %location %ex%n + + + + + + + + + + + + + \ No newline at end of file Index: core/src/test/resources/FastRollingFileAppenderTest.xml =================================================================== --- core/src/test/resources/FastRollingFileAppenderTest.xml (revision 0) +++ core/src/test/resources/FastRollingFileAppenderTest.xml (working copy) @@ -0,0 +1,21 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %location %ex%n + + + + + + + + + + + + + \ No newline at end of file Index: core/src/test/resources/log4j.dtd =================================================================== --- core/src/test/resources/log4j.dtd (revision 0) +++ core/src/test/resources/log4j.dtd (working copy) @@ -0,0 +1,170 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Index: core/src/test/resources/perf-log4j12-async.xml =================================================================== --- core/src/test/resources/perf-log4j12-async.xml (revision 0) +++ core/src/test/resources/perf-log4j12-async.xml (working copy) @@ -0,0 +1,20 @@ + + + + + + + + + + + + + + + + + + + + \ No newline at end of file Index: core/src/test/resources/perf-log4j12.xml =================================================================== --- core/src/test/resources/perf-log4j12.xml (revision 0) +++ core/src/test/resources/perf-log4j12.xml (working copy) @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + + \ No newline at end of file Index: core/src/test/resources/perf-logback-async.xml =================================================================== --- core/src/test/resources/perf-logback-async.xml (revision 0) +++ core/src/test/resources/perf-logback-async.xml (working copy) @@ -0,0 +1,21 @@ + + + + perftest.log + false + + %d %p %c{1} [%t] %X{aKey} %m %ex%n + false + + + + 262144 + 0 + false + + + + + + + \ No newline at end of file Index: core/src/test/resources/perf-logback.xml =================================================================== --- core/src/test/resources/perf-logback.xml (revision 0) +++ core/src/test/resources/perf-logback.xml (working copy) @@ -0,0 +1,15 @@ + + + + perftest.log + false + + %d %p %c{1} [%t] %X{aKey} %m %ex%n + false + + + + + + + \ No newline at end of file Index: core/src/test/resources/perf1syncFastFile.xml =================================================================== --- core/src/test/resources/perf1syncFastFile.xml (revision 0) +++ core/src/test/resources/perf1syncFastFile.xml (working copy) @@ -0,0 +1,15 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %ex%n + + + + + + + + + Index: core/src/test/resources/perf1syncFile.xml =================================================================== --- core/src/test/resources/perf1syncFile.xml (revision 0) +++ core/src/test/resources/perf1syncFile.xml (working copy) @@ -0,0 +1,15 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %ex%n + + + + + + + + + Index: core/src/test/resources/perf2syncRollFastFile.xml =================================================================== --- core/src/test/resources/perf2syncRollFastFile.xml (revision 0) +++ core/src/test/resources/perf2syncRollFastFile.xml (working copy) @@ -0,0 +1,20 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %ex%n + + + + + + + + + + + + Index: core/src/test/resources/perf2syncRollFile.xml =================================================================== --- core/src/test/resources/perf2syncRollFile.xml (revision 0) +++ core/src/test/resources/perf2syncRollFile.xml (working copy) @@ -0,0 +1,20 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %ex%n + + + + + + + + + + + + Index: core/src/test/resources/perf3PlainNoLoc.xml =================================================================== --- core/src/test/resources/perf3PlainNoLoc.xml (revision 0) +++ core/src/test/resources/perf3PlainNoLoc.xml (working copy) @@ -0,0 +1,15 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %ex%n + + + + + + + + + Index: core/src/test/resources/perf4PlainLocation.xml =================================================================== --- core/src/test/resources/perf4PlainLocation.xml (revision 0) +++ core/src/test/resources/perf4PlainLocation.xml (working copy) @@ -0,0 +1,15 @@ + + + + + + %d %p %c{1.} %C %location %line [%t] %X{aKey} %m %ex%n + + + + + + + + + Index: core/src/test/resources/perf5AsyncApndNoLoc.xml =================================================================== --- core/src/test/resources/perf5AsyncApndNoLoc.xml (revision 0) +++ core/src/test/resources/perf5AsyncApndNoLoc.xml (working copy) @@ -0,0 +1,18 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %ex%n + + + + + + + + + + + + Index: core/src/test/resources/perf6AsyncApndLoc.xml =================================================================== --- core/src/test/resources/perf6AsyncApndLoc.xml (revision 0) +++ core/src/test/resources/perf6AsyncApndLoc.xml (working copy) @@ -0,0 +1,18 @@ + + + + + + %d %p %c{1.} %C %location %line [%t] %X{aKey} %m %ex%n + + + + + + + + + + + + Index: core/src/test/resources/perf7MixedNoLoc.xml =================================================================== --- core/src/test/resources/perf7MixedNoLoc.xml (revision 0) +++ core/src/test/resources/perf7MixedNoLoc.xml (working copy) @@ -0,0 +1,15 @@ + + + + + + %d %p %c{1.} [%t] %X{aKey} %m %ex%n + + + + + + + + + Index: core/src/test/resources/perf8MixedLoc.xml =================================================================== --- core/src/test/resources/perf8MixedLoc.xml (revision 0) +++ core/src/test/resources/perf8MixedLoc.xml (working copy) @@ -0,0 +1,15 @@ + + + + + + %d %p %c{1.} %C %location %line [%t] %X{aKey} %m %ex%n + + + + + + + + + \ No newline at end of file Index: log4j-async/pom.xml =================================================================== --- log4j-async/pom.xml (revision 1467676) +++ log4j-async/pom.xml (working copy) @@ -1,249 +0,0 @@ - - - - 4.0.0 - - log4j - org.apache.logging.log4j - 2.0-beta5-SNAPSHOT - ../ - - - org.apache.logging.log4j - log4j-async - jar - Apache Log4j Async - Log4j 2.0 Asynchronous Loggers for Low Latency Logging - - UTF-8 - ${basedir}/.. - Asynchronous Logging Documentation - /log4j-async - - - - org.apache.logging.log4j - log4j-api - - - org.apache.logging.log4j - log4j-core - - - org.apache.logging.log4j - log4j-core - test-jar - test - - - com.lmax - disruptor - 3.0.0.beta3 - - - org.apache.logging.log4j - log4j-1.2-api - test - - - org.slf4j - slf4j-api - test - - - ch.qos.logback - logback-core - test - - - ch.qos.logback - logback-classic - test - - - junit - junit - test - - - - - - org.apache.maven.plugins - maven-jar-plugin - - - - test-jar - - - - - - org.apache.maven.plugins - maven-surefire-plugin - - always - - - - org.codehaus.mojo - exec-maven-plugin - 1.2.1 - - - process-classes - - java - - - - - org.apache.logging.log4j.core.config.plugins.PluginManager - - ${project.build.outputDirectory} - org.apache.logging.log4j.async - - - - - - - - - org.apache.maven.plugins - maven-changes-plugin - ${changes.plugin.version} - - - - changes-report - - - - - %URL%/show_bug.cgi?id=%ISSUE% - true - - - - org.apache.maven.plugins - maven-checkstyle-plugin - 2.7 - - - ${log4jParentDir}/checkstyle.xml - ${log4jParentDir}/checkstyle-suppressions.xml - false - basedir=${basedir} - licensedir=${log4jParentDir}/checkstyle-header.txt - - - - org.apache.maven.plugins - maven-javadoc-plugin - ${javadoc.plugin.version} - - Copyright © {inceptionYear}-{currentYear} {organizationName}. All Rights Reserved. Apache Logging, Apache Log4j, Log4j, Apache, the Apache feather logo, and the - Apache Logging project logo are trademarks of The Apache Software Foundation. - - false - true - - - issue - a - JIRA issue: - - - doubt - a - Troublesome: - - - compare - a - Compare with: - - - - - - non-aggregate - - javadoc - - - - - - - org.apache.maven.plugins - maven-jxr-plugin - 2.3 - - - non-aggregate - - jxr - - - - aggregate - - aggregate - - - - - - org.apache.maven.plugins - maven-pmd-plugin - ${pmd.plugin.version} - - 1.5 - - - - org.codehaus.mojo - cobertura-maven-plugin - 2.2 - - - - - - - - - - - \ No newline at end of file Index: log4j-async/src/main/java/org/apache/logging/log4j/async/AsyncLogger.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/AsyncLogger.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/AsyncLogger.java (working copy) @@ -1,245 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import java.util.Map; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Executors; - -import org.apache.logging.log4j.Level; -import org.apache.logging.log4j.Marker; -import org.apache.logging.log4j.ThreadContext; -import org.apache.logging.log4j.core.Logger; -import org.apache.logging.log4j.core.LoggerContext; -import org.apache.logging.log4j.core.config.Property; -import org.apache.logging.log4j.core.impl.Log4jLogEvent; -import org.apache.logging.log4j.message.Message; -import org.apache.logging.log4j.message.MessageFactory; -import org.apache.logging.log4j.status.StatusLogger; - -import com.lmax.disruptor.BlockingWaitStrategy; -import com.lmax.disruptor.EventHandler; -import com.lmax.disruptor.ExceptionHandler; -import com.lmax.disruptor.RingBuffer; -import com.lmax.disruptor.SleepingWaitStrategy; -import com.lmax.disruptor.WaitStrategy; -import com.lmax.disruptor.YieldingWaitStrategy; -import com.lmax.disruptor.dsl.Disruptor; -import com.lmax.disruptor.dsl.ProducerType; -import com.lmax.disruptor.util.Util; - -/** - * AsyncLogger is a logger designed for high throughput and low latency logging. - * It does not perform any I/O in the calling (application) thread, but instead - * hands off the work to another thread as soon as possible. The actual logging - * is performed in the background thread. It uses the LMAX Disruptor library for - * inter-thread communication. (http://lmax-exchange.github.com/disruptor/) - *

- * To use AsyncLogger, specify the System property - * -DLog4jContextSelector=log4j.async.AsyncLoggerContextSelector before you - * obtain a Logger, and all Loggers returned by LogManager.getLogger will be - * AsyncLoggers. - *

- * Note that for performance reasons, this logger does not support source - * location. Any %class, %location or %line conversion patterns in your - * log4j.xml configuration will produce either a "?" character or no output at - * all. - *

- * For best performance, use AsyncLogger with the FastFileAppender or - * FastRollingFileAppender, with immediateFlush=false. These appenders have - * built-in support for the batching mechanism used by the Disruptor library, - * and they will flush to disk at the end of each batch. This means that even - * with immediateFlush=false, there will never be any items left in the buffer; - * all log events will all be written to disk in a very efficient manner. - */ -public class AsyncLogger extends Logger { // depends on LOG4J2-151 - private static final StatusLogger LOGGER = StatusLogger.getLogger(); - private static volatile Disruptor disruptor; - private static Clock clock = ClockFactory.getClock(); - - private static ExecutorService executor = Executors - .newSingleThreadExecutor(); - private ThreadLocal threadlocalInfo = new ThreadLocal(); - - static { - int ringBufferSize = calculateRingBufferSize(); - - WaitStrategy waitStrategy = createWaitStrategy(); - disruptor = new Disruptor( - RingBufferLogEvent.FACTORY, ringBufferSize, executor, - ProducerType.MULTI, waitStrategy); - EventHandler[] handlers = new RingBufferLogEventHandler[] { new RingBufferLogEventHandler() }; - disruptor.handleExceptionsWith(getExceptionHandler()); - disruptor.handleEventsWith(handlers); - - LOGGER.debug( - "Starting AsyncLogger disruptor with ringbuffer size {}...", - disruptor.getRingBuffer().getBufferSize()); - disruptor.start(); - } - - private static int calculateRingBufferSize() { - String userPreferredRBSize = System.getProperty( - "AsyncLogger.RingBufferSize", "256000"); - int ringBufferSize = 256000; // default - try { - int size = Integer.parseInt(userPreferredRBSize); - if (size < 128) { - size = 128; - LOGGER.warn( - "Invalid RingBufferSize {}, using minimum size 128.", - userPreferredRBSize); - } - ringBufferSize = size; - } catch (Exception ex) { - LOGGER.warn("Invalid RingBufferSize {}, using default size.", - userPreferredRBSize); - } - return Util.ceilingNextPowerOfTwo(ringBufferSize); - } - - private static WaitStrategy createWaitStrategy() { - String strategy = System.getProperty("AsyncLogger.WaitStrategy"); - LOGGER.debug("property AsyncLogger.WaitStrategy={}", strategy); - if ("Sleep".equals(strategy)) { - LOGGER.debug("disruptor event handler uses SleepingWaitStrategy"); - return new SleepingWaitStrategy(); - } else if ("Yield".equals(strategy)) { - LOGGER.debug("disruptor event handler uses YieldingWaitStrategy"); - return new YieldingWaitStrategy(); - } else if ("Block".equals(strategy)) { - LOGGER.debug("disruptor event handler uses BlockingWaitStrategy"); - return new BlockingWaitStrategy(); - } - LOGGER.debug("disruptor event handler uses SleepingWaitStrategy"); - return new SleepingWaitStrategy(); - } - - private static ExceptionHandler getExceptionHandler() { - String cls = System.getProperty("AsyncLogger.ExceptionHandler"); - if (cls == null) { - LOGGER.debug("No AsyncLogger.ExceptionHandler specified"); - return null; - } - try { - @SuppressWarnings("unchecked") - Class klass = (Class) Class - .forName(cls); - ExceptionHandler result = klass.newInstance(); - LOGGER.debug("AsyncLogger.ExceptionHandler=" + result); - return result; - } catch (Exception ignored) { - LOGGER.debug( - "AsyncLogger.ExceptionHandler not set: error creating " - + cls + ": ", ignored); - return null; - } - } - - private static class Info { - RingBufferLogEventTranslator translator; - String cachedThreadName; - } - - public AsyncLogger(LoggerContext context, String name, - MessageFactory messageFactory) { - super(context, name, messageFactory); - } - - @Override - public void log(Marker marker, String fqcn, Level level, Message data, - Throwable t) { - Info info = threadlocalInfo.get(); - if (info == null) { - info = new Info(); - info.translator = new RingBufferLogEventTranslator(); - info.cachedThreadName = Thread.currentThread().getName(); - threadlocalInfo.set(info); - } - - Boolean includeLocation = config.loggerConfig.isIncludeLocation(); - info.translator.setValues(this, getName(), marker, fqcn, level, data, - t, // - - // config properties are taken care of in the EventHandler - // thread in the #actualAsyncLog method - - // needs shallow copy to be fast (LOG4J2-154) - ThreadContext.getImmutableContext(),// - - // needs shallow copy to be fast (LOG4J2-154) - ThreadContext.getImmutableStack(), // - - // Thread.currentThread().getName(), // - info.cachedThreadName, // - - // location: very expensive operation. LOG4J2-153: - // Only include if "includeLocation=true" is specified, - // exclude if not specified or if "false" was specified. - includeLocation != null && includeLocation ? location(fqcn) - : null, - - // System.currentTimeMillis()); - // CoarseCachedClock: 20% faster than system clock, 16ms gaps - // CachedClock: 10% faster than system clock, smaller gaps - clock.currentTimeMillis()); - - disruptor.publishEvent(info.translator); - } - - private StackTraceElement location(String fqcnOfLogger) { - return Log4jLogEvent.calcLocation(fqcnOfLogger); - } - - /** - * This method is called by the EventHandler that processes the - * RingBufferLogEvent in a separate thread. - * - * @param event the event to log - */ - public void actualAsyncLog(RingBufferLogEvent event) { - Map properties = config.loggerConfig.getProperties(); - event.mergePropertiesIntoContextMap(properties, - config.config.getSubst()); - config.logEvent(event); - } - - public static void stop() { - Disruptor temp = disruptor; - - // Must guarantee that publishing to the RingBuffer has stopped - // before we call disruptor.shutdown() - disruptor = null; // client code fails with NPE if log after stop = OK - temp.shutdown(); - - // wait up to 10 seconds for the ringbuffer to drain - RingBuffer ringBuffer = temp.getRingBuffer(); - for (int i = 0; i < 20; i++) { - if (ringBuffer.hasAvailableCapacity(ringBuffer.getBufferSize())) { - break; - } - try { - Thread.sleep(500); // give ringbuffer some time to drain... - } catch (InterruptedException e) { - } - } - executor.shutdown(); // finally, kill the processor thread - } - -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/AsyncLoggerContext.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/AsyncLoggerContext.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/AsyncLoggerContext.java (working copy) @@ -1,57 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import java.net.URI; - -import org.apache.logging.log4j.core.Logger; -import org.apache.logging.log4j.core.LoggerContext; -import org.apache.logging.log4j.message.MessageFactory; - -// depends on LOG4J2-151 -public class AsyncLoggerContext extends LoggerContext { - - public AsyncLoggerContext(String name) { - super(name); - } - - public AsyncLoggerContext(String name, Object externalContext) { - super(name, externalContext); - } - - public AsyncLoggerContext(String name, Object externalContext, - URI configLocn) { - super(name, externalContext, configLocn); - } - - public AsyncLoggerContext(String name, Object externalContext, - String configLocn) { - super(name, externalContext, configLocn); - } - - // @Override - protected Logger newInstance(LoggerContext ctx, String name, - MessageFactory messageFactory) { - return new AsyncLogger(ctx, name, messageFactory); - } - - @Override - public void stop() { - AsyncLogger.stop(); - super.stop(); - } -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/AsyncLoggerContextSelector.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/AsyncLoggerContextSelector.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/AsyncLoggerContextSelector.java (working copy) @@ -1,48 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import java.net.URI; -import java.util.ArrayList; -import java.util.Collections; -import java.util.List; - -import org.apache.logging.log4j.core.LoggerContext; -import org.apache.logging.log4j.core.selector.ContextSelector; - -public class AsyncLoggerContextSelector implements ContextSelector { - - private static final AsyncLoggerContext context = new AsyncLoggerContext( - "Default"); - - public LoggerContext getContext(String fqcn, ClassLoader loader, - boolean currentContext) { - return context; - } - - public List getLoggerContexts() { - List list = new ArrayList(); - list.add(context); - return Collections.unmodifiableList(list); - } - - public LoggerContext getContext(String fqcn, ClassLoader loader, - boolean currentContext, URI configLocation) { - return context; - } - -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/CachedClock.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/CachedClock.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/CachedClock.java (working copy) @@ -1,56 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import com.lmax.disruptor.util.Util; - - -public class CachedClock implements Clock { - private static CachedClock instance = new CachedClock(); - private volatile long millis = System.currentTimeMillis(); - private volatile short count = 0; - private final Thread updater = new Thread("Clock Updater Thread") { - public void run() { - while (true) { - long time = System.currentTimeMillis(); - millis = time; - Util.getUnsafe().park(true, time + 1); // abs (millis) - // Util.getUnsafe().park(false, 1000 * 1000);// relative(nanos) - } - } - }; - - public static CachedClock instance() { - return instance; - } - - private CachedClock() { - updater.setDaemon(true); - updater.start(); - } - - // @Override - public long currentTimeMillis() { - - // improve granularity: also update time field every 1024 calls. - // (the bit fiddling means we don't need to worry about overflows) - if ((++count & 0x3FF) == 0x3FF) { - millis = System.currentTimeMillis(); - } - return millis; - } -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/Clock.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/Clock.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/Clock.java (working copy) @@ -1,21 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -public interface Clock { - long currentTimeMillis(); -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/ClockFactory.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/ClockFactory.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/ClockFactory.java (working copy) @@ -1,57 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import org.apache.logging.log4j.status.StatusLogger; - -public class ClockFactory { - - public static final String PROPERTY_NAME = "AsyncLogger.Clock"; - //private static final Clock clock = createClock(); - - public static Clock getClock() { - return createClock(); - } - - private static Clock createClock() { - final StatusLogger LOGGER = StatusLogger.getLogger(); - String userRequest = System.getProperty(PROPERTY_NAME); - if (userRequest == null || "SystemClock".equals(userRequest)) { - LOGGER.debug("Using default SystemClock for timestamps"); - return new SystemClock(); - } - if ("org.apache.logging.log4j.async.CachedClock".equals(userRequest) - || "CachedClock".equals(userRequest)) { - LOGGER.debug("Using specified CachedClock for timestamps"); - return CachedClock.instance(); - } - if ("org.apache.logging.log4j.async.CoarseCachedClock" - .equals(userRequest) || "CoarseCachedClock".equals(userRequest)) { - LOGGER.debug("Using specified CoarseCachedClock for timestamps"); - return CoarseCachedClock.instance(); - } - try { - Clock result = (Clock) Class.forName(userRequest).newInstance(); - LOGGER.debug("Using {} for timestamps", userRequest); - return result; - } catch (Exception e) { - String fmt = "Could not create {}: {}, using default SystemClock for timestamps"; - LOGGER.error(fmt, userRequest, e); - return new SystemClock(); - } - } -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/CoarseCachedClock.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/CoarseCachedClock.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/CoarseCachedClock.java (working copy) @@ -1,53 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import com.lmax.disruptor.util.Util; - - -/** - * This Clock implementation is similar to CachedClock. It is slightly faster at - * the cost of some accuracy. - */ -public class CoarseCachedClock implements Clock { - private static CoarseCachedClock instance = new CoarseCachedClock(); - private volatile long millis = System.currentTimeMillis(); - private final Thread updater = new Thread("Clock Updater Thread") { - public void run() { - while (true) { - long time = System.currentTimeMillis(); - millis = time; - Util.getUnsafe().park(true, time + 1); // abs (millis) - // Util.getUnsafe().park(false, 1000 * 1000);// relative(nanos) - } - } - }; - - public static CoarseCachedClock instance() { - return instance; - } - - private CoarseCachedClock() { - updater.setDaemon(true); - updater.start(); - } - - // @Override - public long currentTimeMillis() { - return millis; - } -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/RingBufferLogEvent.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/RingBufferLogEvent.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/RingBufferLogEvent.java (working copy) @@ -1,207 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import java.util.HashMap; -import java.util.Map; - -import org.apache.logging.log4j.Level; -import org.apache.logging.log4j.Marker; -import org.apache.logging.log4j.ThreadContext.ContextStack; -import org.apache.logging.log4j.core.LogEvent; -import org.apache.logging.log4j.core.config.Property; -import org.apache.logging.log4j.core.lookup.StrSubstitutor; -import org.apache.logging.log4j.message.Message; -import org.apache.logging.log4j.message.SimpleMessage; - -import com.lmax.disruptor.EventFactory; - -/** - * When the Disruptor is started, the RingBuffer is populated with event - * objects. These objects are then re-used during the life of the RingBuffer. - */ -public class RingBufferLogEvent implements LogEvent { - private static final long serialVersionUID = 8462119088943934758L; - - public static final Factory FACTORY = new Factory(); - - /** - * Creates the events that will be put in the RingBuffer. - */ - private static class Factory implements EventFactory { - // @Override - public RingBufferLogEvent newInstance() { - return new RingBufferLogEvent(); - } - } - - private AsyncLogger asyncLogger; - private String loggerName; - private Marker marker; - private String fqcn; - private Level level; - private Message message; - private Throwable thrown; - private Map contextMap; - private ContextStack contextStack; - private String threadName; - private StackTraceElement location; - private long currentTimeMillis; - private boolean endOfBatch; - private boolean includeLocation; - - public void setValues(AsyncLogger asyncLogger, String loggerName, - Marker marker, String fqcn, Level level, Message data, Throwable t, - Map map, ContextStack contextStack, - String threadName, StackTraceElement location, - long currentTimeMillis) { - this.asyncLogger = asyncLogger; - this.loggerName = loggerName; - this.marker = marker; - this.fqcn = fqcn; - this.level = level; - this.message = data; - this.thrown = t; - this.contextMap = map; - this.contextStack = contextStack; - this.threadName = threadName; - this.location = location; - this.currentTimeMillis = currentTimeMillis; - } - - /** - * Event processor that reads the event from the ringbuffer can call this - * method. - * - * @param endOfBatch flag to indicate if this is the last event in a batch - * from the RingBuffer - */ - public void execute(boolean endOfBatch) { - this.endOfBatch = endOfBatch; - asyncLogger.actualAsyncLog(this); - } - - /** - * Returns {@code true} if this event is the end of a batch, {@code false} - * otherwise. - * - * @return {@code true} if this event is the end of a batch, {@code false} - * otherwise - */ - public boolean isEndOfBatch() { - return endOfBatch; - } - - public void setEndOfBatch(boolean endOfBatch) { - this.endOfBatch = endOfBatch; - } - - public boolean isIncludeLocation() { - return includeLocation; - } - - public void setIncludeLocation(boolean includeLocation) { - this.includeLocation = includeLocation; - } - - // @Override - public String getLoggerName() { - return loggerName; - } - - // @Override - public Marker getMarker() { - return marker; - } - - // @Override - public String getFQCN() { - return fqcn; - } - - // @Override - public Level getLevel() { - return level; - } - - // @Override - public Message getMessage() { - if (message == null) { - message = new SimpleMessage(""); - } - return message; - } - - // @Override - public Throwable getThrown() { - return thrown; - } - - // @Override - public Map getContextMap() { - return contextMap; - } - - // @Override - public ContextStack getContextStack() { - return contextStack; - } - - // @Override - public String getThreadName() { - return threadName; - } - - // @Override - public StackTraceElement getSource() { - return location; - } - - // @Override - public long getMillis() { - return currentTimeMillis; - } - - /** - * Merges the contents of the specified map into the contextMap, after - * replacing any variables in the property values with the - * StrSubstitutor-supplied actual values. - * - * @param properties configured properties - * @param strSubstitutor used to lookup values of variables in properties - */ - public void mergePropertiesIntoContextMap( - Map properties, StrSubstitutor strSubstitutor) { - if (properties == null) { - return; // nothing to do - } - - Map map = (contextMap == null) ? new HashMap() - : new HashMap(contextMap); - - for (Map.Entry entry : properties.entrySet()) { - Property prop = entry.getKey(); - if (map.containsKey(prop.getName())) { - continue; // contextMap overrides config properties - } - String value = entry.getValue() ? strSubstitutor.replace(prop - .getValue()) : prop.getValue(); - map.put(prop.getName(), value); - } - contextMap = map; - } -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/RingBufferLogEventHandler.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/RingBufferLogEventHandler.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/RingBufferLogEventHandler.java (working copy) @@ -1,36 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import com.lmax.disruptor.EventHandler; - -/** - * This event handler gets passed messages from the RingBuffer as they become - * available. Processing of these messages is done in a separate thread, - * controlled by the {@code Executor} passed to the {@code Disruptor} - * constructor. - */ -public class RingBufferLogEventHandler implements - EventHandler { - - // @Override - public void onEvent(RingBufferLogEvent event, long sequence, - boolean endOfBatch) throws Exception { - event.execute(endOfBatch); - } - -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/RingBufferLogEventTranslator.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/RingBufferLogEventTranslator.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/RingBufferLogEventTranslator.java (working copy) @@ -1,70 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import java.util.Map; - -import org.apache.logging.log4j.Level; -import org.apache.logging.log4j.Marker; -import org.apache.logging.log4j.ThreadContext.ContextStack; -import org.apache.logging.log4j.message.Message; - -import com.lmax.disruptor.EventTranslator; - -public class RingBufferLogEventTranslator implements - EventTranslator { - - private AsyncLogger asyncLogger; - private String loggerName; - private Marker marker; - private String fqcn; - private Level level; - private Message message; - private Throwable thrown; - private Map contextMap; - private ContextStack contextStack; - private String threadName; - private StackTraceElement location; - private long currentTimeMillis; - - // @Override - public void translateTo(RingBufferLogEvent event, long sequence) { - event.setValues(asyncLogger, loggerName, marker, fqcn, level, message, - thrown, contextMap, contextStack, threadName, location, - currentTimeMillis); - } - - public void setValues(AsyncLogger asyncLogger, String loggerName, - Marker marker, String fqcn, Level level, Message message, - Throwable thrown, Map contextMap, - ContextStack contextStack, String threadName, - StackTraceElement location, long currentTimeMillis) { - this.asyncLogger = asyncLogger; - this.loggerName = loggerName; - this.marker = marker; - this.fqcn = fqcn; - this.level = level; - this.message = message; - this.thrown = thrown; - this.contextMap = contextMap; - this.contextStack = contextStack; - this.threadName = threadName; - this.location = location; - this.currentTimeMillis = currentTimeMillis; - } - -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/SystemClock.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/SystemClock.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/SystemClock.java (working copy) @@ -1,26 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -public class SystemClock implements Clock { - - // @Override - public long currentTimeMillis() { - return System.currentTimeMillis(); - } - -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastFileAppender.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastFileAppender.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastFileAppender.java (working copy) @@ -1,159 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.appender; - -import java.util.HashMap; -import java.util.Map; - -import org.apache.logging.log4j.core.Filter; -import org.apache.logging.log4j.core.Layout; -import org.apache.logging.log4j.core.LogEvent; -import org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender; -import org.apache.logging.log4j.core.config.Configuration; -import org.apache.logging.log4j.core.config.plugins.Plugin; -import org.apache.logging.log4j.core.config.plugins.PluginAttr; -import org.apache.logging.log4j.core.config.plugins.PluginConfiguration; -import org.apache.logging.log4j.core.config.plugins.PluginElement; -import org.apache.logging.log4j.core.config.plugins.PluginFactory; -import org.apache.logging.log4j.core.layout.PatternLayout; -import org.apache.logging.log4j.core.net.Advertiser; - -/** - * File Appender. - */ -@Plugin(name = "FastFile", type = "Core", elementType = "appender", printObject = true) -public final class FastFileAppender extends AbstractOutputStreamAppender { - - private final String fileName; - private Object advertisement; - private final Advertiser advertiser; - - private FastFileAppender(String name, Layout layout, Filter filter, - FastFileManager manager, String filename, boolean handleException, - boolean immediateFlush, Advertiser advertiser) { - super(name, layout, filter, handleException, immediateFlush, manager); - if (advertiser != null) { - Map configuration = new HashMap( - layout.getContentFormat()); - configuration.putAll(manager.getContentFormat()); - configuration.put("contentType", layout.getContentType()); - configuration.put("name", name); - advertisement = advertiser.advertise(configuration); - } - this.fileName = filename; - this.advertiser = advertiser; - } - - @Override - public void stop() { - super.stop(); - if (advertiser != null) { - advertiser.unadvertise(advertisement); - } - } - - /** - * Write the log entry rolling over the file when required. - * - * @param event The LogEvent. - */ - @Override - public void append(LogEvent event) { - - // Leverage the nice batching behaviour of async Loggers/Appenders: - // we can signal the file manager that it needs to flush the buffer - // to disk at the end of a batch. - // From a user's point of view, this means that all log events are - // _always_ available in the log file, without incurring the overhead - // of immediateFlush=true. - ((FastFileManager) getManager()).setEndOfBatch(event.isEndOfBatch()); - super.append(event); - } - - /** - * Returns the file name this appender is associated with. - * - * @return The File name. - */ - public String getFileName() { - return this.fileName; - } - - // difference from standard File Appender: - // locking is not supported and buffering cannot be switched off - /** - * Create a File Appender. - * - * @param fileName The name and path of the file. - * @param append "True" if the file should be appended to, "false" if it - * should be overwritten. The default is "true". - * @param name The name of the Appender. - * @param immediateFlush "true" if the contents should be flushed on every - * write, "false" otherwise. The default is "true". - * @param suppress "true" if exceptions should be hidden from the - * application, "false" otherwise. The default is "true". - * @param layout The layout to use to format the event. If no layout is - * provided the default PatternLayout will be used. - * @param filter The filter, if any, to use. - * @param advertise "true" if the appender configuration should be - * advertised, "false" otherwise. - * @param advertiseURI The advertised URI which can be used to retrieve the - * file contents. - * @param config The Configuration. - * @return The FileAppender. - */ - @PluginFactory - public static FastFileAppender createAppender( - @PluginAttr("fileName") String fileName, - @PluginAttr("append") String append, - @PluginAttr("name") String name, - @PluginAttr("immediateFlush") String immediateFlush, - @PluginAttr("suppressExceptions") String suppress, - @PluginElement("layout") Layout layout, - @PluginElement("filters") final Filter filter, - @PluginAttr("advertise") final String advertise, - @PluginAttr("advertiseURI") final String advertiseURI, - @PluginConfiguration final Configuration config) { - - boolean isAppend = append == null ? true : Boolean.valueOf(append); - boolean isFlush = immediateFlush == null ? true : Boolean.valueOf(immediateFlush); - boolean handleExceptions = suppress == null ? true : Boolean.valueOf(suppress); - boolean isAdvertise = advertise == null ? false : Boolean.valueOf(advertise); - - if (name == null) { - LOGGER.error("No name provided for FileAppender"); - return null; - } - - if (fileName == null) { - LOGGER.error("No filename provided for FileAppender with name " - + name); - return null; - } - - FastFileManager manager = FastFileManager.getFileManager(fileName, - isAppend, isFlush, advertiseURI); - if (manager == null) { - return null; - } - if (layout == null) { - layout = PatternLayout.createLayout(null, null, null, null); - } - return new FastFileAppender(name, layout, filter, manager, fileName, - handleExceptions, isFlush, isAdvertise ? config.getAdvertiser() : null); - } -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastFileManager.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastFileManager.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastFileManager.java (working copy) @@ -1,205 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.appender; - -import java.io.File; -import java.io.IOException; -import java.io.OutputStream; -import java.io.RandomAccessFile; -import java.nio.ByteBuffer; -import java.util.HashMap; -import java.util.Map; - -import org.apache.logging.log4j.core.appender.AppenderRuntimeException; -import org.apache.logging.log4j.core.appender.ManagerFactory; -import org.apache.logging.log4j.core.appender.OutputStreamManager; - -/** - * Manages actual File I/O for File Appenders. - */ -public class FastFileManager extends OutputStreamManager { - - private static final FastFileManagerFactory factory = new FastFileManagerFactory(); - - private final boolean isImmediateFlush; - private final String advertiseURI; - private final RandomAccessFile randomAccessFile; - private final ByteBuffer buffer; - private ThreadLocal isEndOfBatch = new ThreadLocal(); - - protected FastFileManager(RandomAccessFile file, String fileName, - OutputStream os, boolean immediateFlush, String advertiseURI) { - super(os, fileName); - this.isImmediateFlush = immediateFlush; - this.randomAccessFile = file; - this.advertiseURI = advertiseURI; - isEndOfBatch.set(Boolean.FALSE); - buffer = ByteBuffer.allocate(256 * 1024); // TODO make configurable? - } - - /** - * Returns the FastFileManager. - * - * @param fileName - * The name of the file to manage. - * @param append - * true if the file should be appended to, false if it should be - * overwritten. - * @param isFlush - * true if the contents should be flushed to disk on every write - * @param advertiseURI the URI to use when advertising the file - * @return A FastFileManager for the File. - */ - public static FastFileManager getFileManager(String fileName, - boolean append, boolean isFlush, String advertiseURI) { - return (FastFileManager) getManager(fileName, new FactoryData(append, - isFlush, advertiseURI), factory); - } - - public Boolean isEndOfBatch() { - return isEndOfBatch.get(); - } - - public void setEndOfBatch(boolean isEndOfBatch) { - this.isEndOfBatch.set(Boolean.valueOf(isEndOfBatch)); - } - - @Override - protected synchronized void write(byte[] bytes, int offset, int length) { - super.write(bytes, offset, length); // writes to dummy output stream - - if (length > buffer.remaining()) { - flush(); - } - buffer.put(bytes, offset, length); - if (isImmediateFlush || isEndOfBatch.get() == Boolean.TRUE) { - flush(); - } - } - - @Override - public void flush() { - buffer.flip(); - try { - randomAccessFile.write(buffer.array(), 0, buffer.limit()); - } catch (IOException ex) { - String msg = "Error writing to RandomAccessFile " + getName(); - throw new AppenderRuntimeException(msg, ex); - } - buffer.clear(); - } - - @Override - public void close() { - flush(); - try { - randomAccessFile.close(); - } catch (final IOException ex) { - LOGGER.error("Unable to close RandomAccessFile " + getName() + ". " + ex); - } - } - - /** - * Returns the name of the File being managed. - * - * @return The name of the File being managed. - */ - public String getFileName() { - return getName(); - } - - private static class DummyOutputStream extends OutputStream { - @Override - public void write(int b) throws IOException { - } - - @Override - public void write(byte[] b, int off, int len) throws IOException { - } - } - - /** - * FileManager's content format is specified by:

- * Key: "fileURI" Value: provided "advertiseURI" param - * @return Map of content format keys supporting FileManager - */ - public Map getContentFormat() { - Map result = new HashMap(super.getContentFormat()); - result.put("fileURI", advertiseURI); - return result; - } - - /** - * Factory Data. - */ - private static class FactoryData { - private final boolean append; - private final boolean immediateFlush; - private final String advertiseURI; - - /** - * Constructor. - * - * @param append - * Append status. - */ - public FactoryData(boolean append, boolean immediateFlush, String advertiseURI) { - this.append = append; - this.immediateFlush = immediateFlush; - this.advertiseURI = advertiseURI; - } - } - - /** - * Factory to create a FastFileManager. - */ - private static class FastFileManagerFactory implements - ManagerFactory { - - /** - * Create a FastFileManager. - * - * @param name - * The name of the File. - * @param data - * The FactoryData - * @return The FastFileManager for the File. - */ - public FastFileManager createManager(String name, FactoryData data) { - File file = new File(name); - final File parent = file.getParentFile(); - if (null != parent && !parent.exists()) { - parent.mkdirs(); - } - if (!data.append) { - file.delete(); - } - - OutputStream os = new DummyOutputStream(); - RandomAccessFile raf; - try { - raf = new RandomAccessFile(name, "rw"); - return new FastFileManager(raf, name, os, data.immediateFlush, - data.advertiseURI); - } catch (Exception ex) { - LOGGER.error("FastFileManager (" + name + ") " + ex); - } - return null; - } - } - -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastRollingFileAppender.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastRollingFileAppender.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastRollingFileAppender.java (working copy) @@ -1,189 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.appender; - -import java.util.HashMap; -import java.util.Map; - -import org.apache.logging.log4j.core.Filter; -import org.apache.logging.log4j.core.Layout; -import org.apache.logging.log4j.core.LogEvent; -import org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender; -import org.apache.logging.log4j.core.appender.rolling.DefaultRolloverStrategy; -import org.apache.logging.log4j.core.appender.rolling.RollingFileManager; -import org.apache.logging.log4j.core.appender.rolling.RolloverStrategy; -import org.apache.logging.log4j.core.appender.rolling.TriggeringPolicy; -import org.apache.logging.log4j.core.config.Configuration; -import org.apache.logging.log4j.core.config.plugins.Plugin; -import org.apache.logging.log4j.core.config.plugins.PluginAttr; -import org.apache.logging.log4j.core.config.plugins.PluginConfiguration; -import org.apache.logging.log4j.core.config.plugins.PluginElement; -import org.apache.logging.log4j.core.config.plugins.PluginFactory; -import org.apache.logging.log4j.core.layout.PatternLayout; -import org.apache.logging.log4j.core.net.Advertiser; - -/** - * An appender that writes to random access files and can roll over at - * intervals. - */ -@Plugin(name = "FastRollingFile", type = "Core", elementType = "appender", printObject = true) -public final class FastRollingFileAppender extends AbstractOutputStreamAppender { - - private final String fileName; - private final String filePattern; - private Object advertisement; - private final Advertiser advertiser; - - private FastRollingFileAppender(String name, Layout layout, Filter filter, - RollingFileManager manager, String fileName, - String filePattern, boolean handleException, - boolean immediateFlush, Advertiser advertiser) { - super(name, layout, filter, handleException, immediateFlush, manager); - if (advertiser != null) { - Map configuration = new HashMap(layout.getContentFormat()); - configuration.put("contentType", layout.getContentType()); - configuration.put("name", name); - advertisement = advertiser.advertise(configuration); - } - this.fileName = fileName; - this.filePattern = filePattern; - this.advertiser = advertiser; - } - - @Override - public void stop() { - super.stop(); - if (advertiser != null) { - advertiser.unadvertise(advertisement); - } - } - - /** - * Write the log entry rolling over the file when required. - - * @param event The LogEvent. - */ - @Override - public void append(final LogEvent event) { - FastRollingFileManager manager = (FastRollingFileManager) getManager(); - manager.checkRollover(event); - - // Leverage the nice batching behaviour of async Loggers/Appenders: - // we can signal the file manager that it needs to flush the buffer - // to disk at the end of a batch. - // From a user's point of view, this means that all log events are - // _always_ available in the log file, without incurring the overhead - // of immediateFlush=true. - manager.setEndOfBatch(event.isEndOfBatch()); - super.append(event); - } - - /** - * Returns the File name for the Appender. - * @return The file name. - */ - public String getFileName() { - return fileName; - } - - /** - * Returns the file pattern used when rolling over. - * @return The file pattern. - */ - public String getFilePattern() { - return filePattern; - } - - /** - * Create a FastRollingFileAppender. - * @param fileName The name of the file that is actively written to. (required). - * @param filePattern The pattern of the file name to use on rollover. (required). - * @param append If true, events are appended to the file. If false, the file - * is overwritten when opened. Defaults to "true" - * @param name The name of the Appender (required). - * @param immediateFlush When true, events are immediately flushed. Defaults to "true". - * @param policy The triggering policy. (required). - * @param strategy The rollover strategy. Defaults to DefaultRolloverStrategy. - * @param layout The layout to use (defaults to the default PatternLayout). - * @param filter The Filter or null. - * @param suppress "true" if exceptions should be hidden from the application, "false" otherwise. - * The default is "true". - * @param advertise "true" if the appender configuration should be advertised, "false" otherwise. - * @param advertiseURI The advertised URI which can be used to retrieve the file contents. - * @param config The Configuration. - * @return A FastRollingFileAppender. - */ - @PluginFactory - public static FastRollingFileAppender createAppender( - @PluginAttr("fileName") final String fileName, - @PluginAttr("filePattern") final String filePattern, - @PluginAttr("append") final String append, - @PluginAttr("name") final String name, - @PluginAttr("immediateFlush") final String immediateFlush, - @PluginElement("policy") final TriggeringPolicy policy, - @PluginElement("strategy") RolloverStrategy strategy, - @PluginElement("layout") Layout layout, - @PluginElement("filter") final Filter filter, - @PluginAttr("suppressExceptions") final String suppress, - @PluginAttr("advertise") final String advertise, - @PluginAttr("advertiseURI") final String advertiseURI, - @PluginConfiguration final Configuration config) { - - final boolean isAppend = append == null ? true : Boolean.valueOf(append); - final boolean handleExceptions = suppress == null ? true : Boolean.valueOf(suppress); - final boolean isFlush = immediateFlush == null ? true : Boolean.valueOf(immediateFlush); - boolean isAdvertise = advertise == null ? false : Boolean.valueOf(advertise); - - if (name == null) { - LOGGER.error("No name provided for FileAppender"); - return null; - } - - if (fileName == null) { - LOGGER.error("No filename was provided for FileAppender with name " + name); - return null; - } - - if (filePattern == null) { - LOGGER.error("No filename pattern provided for FileAppender with name " + name); - return null; - } - - if (policy == null) { - LOGGER.error("A TriggeringPolicy must be provided"); - return null; - } - - if (strategy == null) { - strategy = DefaultRolloverStrategy.createStrategy(null, null, "true", config); - } - - final FastRollingFileManager manager = FastRollingFileManager - .getFastRollingFileManager(fileName, filePattern, isAppend, - isFlush, policy, strategy, advertiseURI); - if (manager == null) { - return null; - } - - if (layout == null) { - layout = PatternLayout.createLayout(null, null, null, null); - } - - return new FastRollingFileAppender(name, layout, filter, manager, fileName, filePattern, - handleExceptions, isFlush, isAdvertise ? config.getAdvertiser() : null); - } -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastRollingFileManager.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastRollingFileManager.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/appender/FastRollingFileManager.java (working copy) @@ -1,195 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.appender; - -import java.io.File; -import java.io.FileNotFoundException; -import java.io.IOException; -import java.io.OutputStream; -import java.io.RandomAccessFile; -import java.nio.ByteBuffer; - -import org.apache.logging.log4j.core.appender.AppenderRuntimeException; -import org.apache.logging.log4j.core.appender.ManagerFactory; -import org.apache.logging.log4j.core.appender.rolling.RollingFileManager; -import org.apache.logging.log4j.core.appender.rolling.RolloverStrategy; -import org.apache.logging.log4j.core.appender.rolling.TriggeringPolicy; - -public class FastRollingFileManager extends RollingFileManager { - private static FastRollingFileManagerFactory factory = new FastRollingFileManagerFactory(); - - private final boolean isImmediateFlush; - private RandomAccessFile randomAccessFile; - private final ByteBuffer buffer; - private ThreadLocal isEndOfBatch = new ThreadLocal(); - - public FastRollingFileManager(RandomAccessFile raf, String fileName, - String pattern, OutputStream os, boolean append, - boolean immediateFlush, long size, long time, TriggeringPolicy policy, - RolloverStrategy strategy, String advertiseURI) { - super(fileName, pattern, os, append, size, time, policy, strategy, advertiseURI); - this.isImmediateFlush = immediateFlush; - this.randomAccessFile = raf; - isEndOfBatch.set(Boolean.FALSE); - buffer = ByteBuffer.allocate(256 * 1024); // TODO make configurable? - } - - public static FastRollingFileManager getFastRollingFileManager( - String fileName, String filePattern, boolean isAppend, - boolean immediateFlush, TriggeringPolicy policy, - RolloverStrategy strategy, String advertiseURI) { - return (FastRollingFileManager) getManager(fileName, new FactoryData( - filePattern, isAppend, immediateFlush, policy, strategy, - advertiseURI), factory); - } - - public Boolean isEndOfBatch() { - return isEndOfBatch.get(); - } - - public void setEndOfBatch(boolean isEndOfBatch) { - this.isEndOfBatch.set(Boolean.valueOf(isEndOfBatch)); - } - - @Override - protected synchronized void write(byte[] bytes, int offset, int length) { - super.write(bytes, offset, length); // writes to dummy output stream - - if (length > buffer.remaining()) { - flush(); - } - buffer.put(bytes, offset, length); - if (isImmediateFlush || isEndOfBatch.get() == Boolean.TRUE) { - flush(); - } - } - - @Override - protected void createFileAfterRollover() throws IOException { - this.randomAccessFile = new RandomAccessFile(getFileName(), "rw"); - if (isAppend()) { - randomAccessFile.seek(randomAccessFile.length()); - } - } - - @Override - public void flush() { - buffer.flip(); - try { - randomAccessFile.write(buffer.array(), 0, buffer.limit()); - } catch (IOException ex) { - String msg = "Error writing to RandomAccessFile " + getName(); - throw new AppenderRuntimeException(msg, ex); - } - buffer.clear(); - } - - @Override - public void close() { - flush(); - try { - randomAccessFile.close(); - } catch (final IOException ex) { - LOGGER.error("Unable to close RandomAccessFile " + getName() + ". " + ex); - } - } - - /** - * Factory to create a FastRollingFileManager. - */ - private static class FastRollingFileManagerFactory implements - ManagerFactory { - - /** - * Create the FastRollingFileManager. - * - * @param name - * The name of the entity to manage. - * @param data - * The data required to create the entity. - * @return a RollingFileManager. - */ - public FastRollingFileManager createManager(String name, - FactoryData data) { - File file = new File(name); - final File parent = file.getParentFile(); - if (null != parent && !parent.exists()) { - parent.mkdirs(); - } - if (!data.append) { - file.delete(); - } - long size = data.append ? file.length() : 0; - long time = file.lastModified(); - - RandomAccessFile raf; - try { - raf = new RandomAccessFile(name, "rw"); - return new FastRollingFileManager(raf, name, data.pattern, - new DummyOutputStream(), data.append, - data.immediateFlush, size, time, data.policy, - data.strategy, data.advertiseURI); - } catch (FileNotFoundException ex) { - LOGGER.error("FastRollingFileManager (" + name + ") " + ex); - } - return null; - } - } - - private static class DummyOutputStream extends OutputStream { - @Override - public void write(int b) throws IOException { - } - - @Override - public void write(byte[] b, int off, int len) throws IOException { - } - } - - /** - * Factory data. - */ - private static class FactoryData { - private final String pattern; - private final boolean append; - private final boolean immediateFlush; - private final TriggeringPolicy policy; - private final RolloverStrategy strategy; - private final String advertiseURI; - - /** - * Create the data for the factory. - * - * @param pattern - * The pattern. - * @param append - * The append flag. - * @param immediateFlush - */ - public FactoryData(String pattern, boolean append, - boolean immediateFlush, TriggeringPolicy policy, - RolloverStrategy strategy, String advertiseURI) { - this.pattern = pattern; - this.append = append; - this.immediateFlush = immediateFlush; - this.policy = policy; - this.strategy = strategy; - this.advertiseURI = advertiseURI; - } - } - -} Index: log4j-async/src/main/java/org/apache/logging/log4j/async/config/AsyncLoggerConfig.java =================================================================== --- log4j-async/src/main/java/org/apache/logging/log4j/async/config/AsyncLoggerConfig.java (revision 1467676) +++ log4j-async/src/main/java/org/apache/logging/log4j/async/config/AsyncLoggerConfig.java (working copy) @@ -1,354 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.config; - -import java.util.Arrays; -import java.util.List; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Executors; - -import org.apache.logging.log4j.Level; -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.Filter; -import org.apache.logging.log4j.core.LogEvent; -import org.apache.logging.log4j.core.LoggerContext; -import org.apache.logging.log4j.core.LoggerContext.Status; -import org.apache.logging.log4j.core.config.AppenderRef; -import org.apache.logging.log4j.core.config.Configuration; -import org.apache.logging.log4j.core.config.LoggerConfig; -import org.apache.logging.log4j.core.config.Property; -import org.apache.logging.log4j.core.config.plugins.Plugin; -import org.apache.logging.log4j.core.config.plugins.PluginAttr; -import org.apache.logging.log4j.core.config.plugins.PluginConfiguration; -import org.apache.logging.log4j.core.config.plugins.PluginElement; -import org.apache.logging.log4j.core.config.plugins.PluginFactory; -import org.apache.logging.log4j.status.StatusLogger; - -import com.lmax.disruptor.BlockingWaitStrategy; -import com.lmax.disruptor.EventFactory; -import com.lmax.disruptor.EventHandler; -import com.lmax.disruptor.EventTranslator; -import com.lmax.disruptor.ExceptionHandler; -import com.lmax.disruptor.RingBuffer; -import com.lmax.disruptor.SleepingWaitStrategy; -import com.lmax.disruptor.WaitStrategy; -import com.lmax.disruptor.YieldingWaitStrategy; -import com.lmax.disruptor.dsl.Disruptor; -import com.lmax.disruptor.dsl.ProducerType; -import com.lmax.disruptor.util.Util; - -/** - * Asynchronous Logger object that is created via configuration. - */ -@Plugin(name = "asyncLogger", type = "Core", printObject = true) -public class AsyncLoggerConfig extends LoggerConfig { - - private static final Logger LOGGER = StatusLogger.getLogger(); - private static volatile Disruptor disruptor; - private static ExecutorService executor = Executors - .newSingleThreadExecutor(); - - private ThreadLocal currentLogEvent = new ThreadLocal(); - - /** - * RingBuffer events contain all information necessary to perform the work - * in a separate thread. - */ - private static class RingBufferLog4jEvent { - private AsyncLoggerConfig loggerConfig; - private LogEvent event; - } - - /** - * Factory used to populate the RingBuffer with events. These event objects - * are then re-used during the life of the RingBuffer. - */ - private static final EventFactory FACTORY = new EventFactory() { - @Override - public RingBufferLog4jEvent newInstance() { - return new RingBufferLog4jEvent(); - } - }; - - /** - * Object responsible for passing on data to a specific RingBuffer event. - */ - private final EventTranslator translator = new EventTranslator() { - @Override - public void translateTo(RingBufferLog4jEvent event, long sequence) { - event.event = currentLogEvent.get(); - event.loggerConfig = AsyncLoggerConfig.this; - } - }; - - /** - * EventHandler performs the work in a separate thread. - */ - private static class RingBufferLog4jEventHandler implements - EventHandler { - @Override - public void onEvent(RingBufferLog4jEvent event, long sequence, - boolean endOfBatch) throws Exception { - event.loggerConfig.asyncCallAppenders(event.event, endOfBatch); - } - } - - /** - * Default constructor. - */ - public AsyncLoggerConfig() { - super(); - } - - /** - * Constructor that sets the name, level and additive values. - * - * @param name The Logger name. - * @param level The Level. - * @param additive true if the Logger is additive, false otherwise. - */ - public AsyncLoggerConfig(final String name, final Level level, - final boolean additive) { - super(name, level, additive); - } - - protected AsyncLoggerConfig(final String name, - final List appenders, final Filter filter, - final Level level, final boolean additive, - final Property[] properties, final Configuration config, - final boolean includeLocation) { - super(name, appenders, filter, level, additive, properties, config, - includeLocation); - } - - /** - * Passes on the event to a separate thread that will call - * {@link #asyncCallAppenders(LogEvent)}. - */ - @Override - protected void callAppenders(LogEvent event) { - // populate lazily initialized fields - event.getSource(); - event.getThreadName(); - - // pass on the event to a separate thread - currentLogEvent.set(event); - disruptor.publishEvent(translator); - } - - /** Called by RingBufferLog4jEventHandler. */ - private void asyncCallAppenders(LogEvent event, boolean endOfBatch) { - event.setEndOfBatch(endOfBatch); - super.callAppenders(event); - } - - @Override - public void startFilter() { - if (disruptor == null) { - int ringBufferSize = calculateRingBufferSize(); - WaitStrategy waitStrategy = createWaitStrategy(); - disruptor = new Disruptor(FACTORY, - ringBufferSize, executor, ProducerType.MULTI, waitStrategy); - EventHandler[] handlers = new RingBufferLog4jEventHandler[] { new RingBufferLog4jEventHandler() }; - disruptor.handleExceptionsWith(getExceptionHandler()); - disruptor.handleEventsWith(handlers); - - LOGGER.debug( - "Starting AsyncLoggerConfig disruptor with ringbuffer size {}...", - disruptor.getRingBuffer().getBufferSize()); - disruptor.start(); - } - super.startFilter(); - } - - private WaitStrategy createWaitStrategy() { - String strategy = System.getProperty("AsyncLoggerConfig.WaitStrategy"); - LOGGER.debug("property AsyncLoggerConfig.WaitStrategy={}", strategy); - if ("Sleep".equals(strategy)) { - LOGGER.debug("disruptor event handler uses SleepingWaitStrategy"); - return new SleepingWaitStrategy(); - } else if ("Yield".equals(strategy)) { - LOGGER.debug("disruptor event handler uses YieldingWaitStrategy"); - return new YieldingWaitStrategy(); - } else if ("Block".equals(strategy)) { - LOGGER.debug("disruptor event handler uses BlockingWaitStrategy"); - return new BlockingWaitStrategy(); - } - LOGGER.debug("disruptor event handler uses SleepingWaitStrategy"); - return new SleepingWaitStrategy(); - } - - private static int calculateRingBufferSize() { - String userPreferredRBSize = System.getProperty( - "AsyncLoggerConfig.RingBufferSize", "256000"); - int ringBufferSize = 256000; // default - try { - int size = Integer.parseInt(userPreferredRBSize); - if (size < 128) { - size = 128; - LOGGER.warn( - "Invalid RingBufferSize {}, using minimum size 128.", - userPreferredRBSize); - } - ringBufferSize = size; - } catch (Exception ex) { - LOGGER.warn("Invalid RingBufferSize {}, using default size.", - userPreferredRBSize); - } - return Util.ceilingNextPowerOfTwo(ringBufferSize); - } - - private static ExceptionHandler getExceptionHandler() { - String cls = System.getProperty("AsyncLoggerConfig.ExceptionHandler"); - if (cls == null) { - LOGGER.debug("No AsyncLoggerConfig.ExceptionHandler specified"); - return null; - } - try { - @SuppressWarnings("unchecked") - Class klass = (Class) Class - .forName(cls); - ExceptionHandler result = klass.newInstance(); - LOGGER.debug("AsyncLoggerConfig.ExceptionHandler=" + result); - return result; - } catch (Exception ignored) { - LOGGER.debug( - "AsyncLoggerConfig.ExceptionHandler not set: error creating " - + cls + ": ", ignored); - return null; - } - } - - @Override - public void stopFilter() { - // only stop disruptor if shutting down logging subsystem - if (LogManager.getContext() instanceof LoggerContext) { - if (((LoggerContext) LogManager.getContext()).getStatus() != Status.STOPPING) { - return; - } - } - Disruptor temp = disruptor; - - // Must guarantee that publishing to the RingBuffer has stopped - // before we call disruptor.shutdown() - disruptor = null; // client code fails with NPE if log after stop = OK - temp.shutdown(); - - // wait up to 10 seconds for the ringbuffer to drain - RingBuffer ringBuffer = temp.getRingBuffer(); - for (int i = 0; i < 20; i++) { - if (ringBuffer.hasAvailableCapacity(ringBuffer.getBufferSize())) { - break; - } - try { - Thread.sleep(500); // give ringbuffer some time to drain... - } catch (InterruptedException e) { - } - } - executor.shutdown(); // finally, kill the processor thread - super.stopFilter(); - } - - /** - * Factory method to create a LoggerConfig. - * - * @param additivity True if additive, false otherwise. - * @param levelName The Level to be associated with the Logger. - * @param loggerName The name of the Logger. - * @param includeLocation "true" if location should be passed downstream - * @param refs An array of Appender names. - * @param properties Properties to pass to the Logger. - * @param config The Configuration. - * @param filter A Filter. - * @return A new LoggerConfig. - */ - @PluginFactory - public static LoggerConfig createLogger( - @PluginAttr("additivity") final String additivity, - @PluginAttr("level") final String levelName, - @PluginAttr("name") final String loggerName, - @PluginAttr("includeLocation") final String includeLocation, - @PluginElement("appender-ref") final AppenderRef[] refs, - @PluginElement("properties") final Property[] properties, - @PluginConfiguration final Configuration config, - @PluginElement("filters") final Filter filter) { - if (loggerName == null) { - LOGGER.error("Loggers cannot be configured without a name"); - return null; - } - - final List appenderRefs = Arrays.asList(refs); - Level level; - try { - level = Level.toLevel(levelName, Level.ERROR); - } catch (final Exception ex) { - LOGGER.error( - "Invalid Log level specified: {}. Defaulting to Error", - levelName); - level = Level.ERROR; - } - final String name = loggerName.equals("root") ? "" : loggerName; - final boolean additive = additivity == null ? true : Boolean - .parseBoolean(additivity); - - return new AsyncLoggerConfig(name, appenderRefs, filter, level, - additive, properties, config, includeLocation(includeLocation)); - } - - // Note: for asynchronous loggers, includeLocation default is FALSE - private static boolean includeLocation(String includeLocationConfigValue) { - if (includeLocationConfigValue == null) { - return false; - } - return Boolean.parseBoolean(includeLocationConfigValue); - } - - /** - * An asynchronous root Logger. - */ - @Plugin(name = "asyncRoot", type = "Core", printObject = true) - public static class RootLogger extends LoggerConfig { - - @PluginFactory - public static LoggerConfig createLogger( - @PluginAttr("additivity") final String additivity, - @PluginAttr("level") final String levelName, - @PluginAttr("includeLocation") final String includeLocation, - @PluginElement("appender-ref") final AppenderRef[] refs, - @PluginElement("properties") final Property[] properties, - @PluginConfiguration final Configuration config, - @PluginElement("filters") final Filter filter) { - final List appenderRefs = Arrays.asList(refs); - Level level; - try { - level = Level.toLevel(levelName, Level.ERROR); - } catch (final Exception ex) { - LOGGER.error( - "Invalid Log level specified: {}. Defaulting to Error", - levelName); - level = Level.ERROR; - } - final boolean additive = additivity == null ? true : Boolean - .parseBoolean(additivity); - - return new AsyncLoggerConfig(LogManager.ROOT_LOGGER_NAME, - appenderRefs, filter, level, additive, properties, config, - includeLocation(includeLocation)); - } - } -} Index: log4j-async/src/site/site.xml =================================================================== --- log4j-async/src/site/site.xml (revision 1467676) +++ log4j-async/src/site/site.xml (working copy) @@ -1,27 +0,0 @@ - - - - - - - - -

- - Index: log4j-async/src/site/xdoc/index.xml =================================================================== --- log4j-async/src/site/xdoc/index.xml (revision 1467676) +++ log4j-async/src/site/xdoc/index.xml (working copy) @@ -1,47 +0,0 @@ - - - - - Asynchronous Logging - Remko Popma - - -
-

- Log4j-async uses LMAX Disruptor - technology to significantly improve the performance of - asynchronous logging, especially in multi-threaded scenarios. -

-
- -
-

- Log4j-async requires at least Java 6 and is dependent on the Log4j 2 - API and implementation, as well as the LMAX disruptor library. -

-
- -
-

- Include this jar, the disruptor-3.0.0 jar, the Log4j 2 API and - implementation jar together. Configure - the logging implementation as required. -

-
- -
\ No newline at end of file Index: log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerContextSelectorTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerContextSelectorTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerContextSelectorTest.java (working copy) @@ -1,53 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import static org.junit.Assert.*; - -import java.util.List; - -import org.apache.logging.log4j.core.LoggerContext; -import org.junit.Test; - -public class AsyncLoggerContextSelectorTest { - - @Test - public void testContextReturnsAsyncLoggerContext() { - AsyncLoggerContextSelector selector = new AsyncLoggerContextSelector(); - LoggerContext context = selector.getContext(null, null, false); - - assertTrue(context instanceof AsyncLoggerContext); - } - - @Test - public void testContext2ReturnsAsyncLoggerContext() { - AsyncLoggerContextSelector selector = new AsyncLoggerContextSelector(); - LoggerContext context = selector.getContext(null, null, false, null); - - assertTrue(context instanceof AsyncLoggerContext); - } - - @Test - public void testLoggerContextsReturnsAsyncLoggerContext() { - AsyncLoggerContextSelector selector = new AsyncLoggerContextSelector(); - List list = selector.getLoggerContexts(); - - assertEquals(1, list.size()); - assertTrue(list.get(0) instanceof AsyncLoggerContext); - } - -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerContextTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerContextTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerContextTest.java (working copy) @@ -1,37 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import static org.junit.Assert.*; - -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.LifeCycle; -import org.apache.logging.log4j.core.LoggerContext; -import org.junit.Test; - -public class AsyncLoggerContextTest { - - @Test - public void testNewInstanceReturnsAsyncLogger() { - Logger logger = new AsyncLoggerContext("a").newInstance( - new LoggerContext("a"), "a", null); - assertTrue(logger instanceof AsyncLogger); - - ((LifeCycle) LogManager.getContext()).stop(); // stop async thread - } -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerLocationTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerLocationTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerLocationTest.java (working copy) @@ -1,70 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import static org.junit.Assert.*; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; - -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.LifeCycle; -import org.apache.logging.log4j.core.config.XMLConfigurationFactory; -import org.apache.logging.log4j.core.helpers.Constants; -import org.junit.AfterClass; -import org.junit.BeforeClass; -import org.junit.Test; - -public class AsyncLoggerLocationTest { - - @BeforeClass - public static void beforeClass() { - System.setProperty(Constants.LOG4J_CONTEXT_SELECTOR, - AsyncLoggerContextSelector.class.getName()); - System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, - "AsyncLoggerLocationTest.xml"); - } - - @AfterClass - public static void afterClass() { - System.setProperty(Constants.LOG4J_CONTEXT_SELECTOR, ""); - } - - @Test - public void testAsyncLogWritesToLog() throws Exception { - File f = new File("target", "AsyncLoggerLocationTest.log"); - // System.out.println(f.getAbsolutePath()); - f.delete(); - Logger log = LogManager.getLogger("com.foo.Bar"); - String msg = "Async logger msg with location"; - log.info(msg); - ((LifeCycle) LogManager.getContext()).stop(); // stop async thread - - BufferedReader reader = new BufferedReader(new FileReader(f)); - String line1 = reader.readLine(); - reader.close(); - f.delete(); - assertNotNull("line1", line1); - assertTrue("line1 correct", line1.contains(msg)); - - String location = "testAsyncLogWritesToLog"; - assertTrue("has location", line1.contains(location)); - } - -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/AsyncLoggerTest.java (working copy) @@ -1,70 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import static org.junit.Assert.*; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; - -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.LifeCycle; -import org.apache.logging.log4j.core.config.XMLConfigurationFactory; -import org.apache.logging.log4j.core.helpers.Constants; -import org.junit.AfterClass; -import org.junit.BeforeClass; -import org.junit.Test; - -public class AsyncLoggerTest { - - @BeforeClass - public static void beforeClass() { - System.setProperty(Constants.LOG4J_CONTEXT_SELECTOR, - AsyncLoggerContextSelector.class.getName()); - System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, - "AsyncLoggerTest.xml"); - } - - @AfterClass - public static void afterClass() { - System.setProperty(Constants.LOG4J_CONTEXT_SELECTOR, ""); - } - - @Test - public void testAsyncLogWritesToLog() throws Exception { - File f = new File("target", "AsyncLoggerTest.log"); - // System.out.println(f.getAbsolutePath()); - f.delete(); - Logger log = LogManager.getLogger("com.foo.Bar"); - String msg = "Async logger msg"; - log.info(msg); - ((LifeCycle) LogManager.getContext()).stop(); // stop async thread - - BufferedReader reader = new BufferedReader(new FileReader(f)); - String line1 = reader.readLine(); - reader.close(); - f.delete(); - assertNotNull("line1", line1); - assertTrue("line1 correct", line1.contains(msg)); - - String location = "testAsyncLogWritesToLog"; - assertTrue("no location", !line1.contains(location)); - } - -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/CachedClockTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/CachedClockTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/CachedClockTest.java (working copy) @@ -1,46 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import static org.junit.Assert.*; - -import org.junit.Test; - -public class CachedClockTest { - - @Test - public void testLessThan17Millis() { - long millis1 = CachedClock.instance().currentTimeMillis(); - long sysMillis = System.currentTimeMillis(); - - long diff = sysMillis - millis1; - - assertTrue("diff too large: " + diff, diff <= 16); - } - - @Test - public void testAfterWaitStillLessThan17Millis() throws Exception { - Thread.sleep(100); - long millis1 = CachedClock.instance().currentTimeMillis(); - long sysMillis = System.currentTimeMillis(); - - long diff = sysMillis - millis1; - - assertTrue("diff too large: " + diff, diff <= 16); - } - -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/ClockFactoryTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/ClockFactoryTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/ClockFactoryTest.java (working copy) @@ -1,80 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import static org.junit.Assert.*; - -import org.junit.Test; - -public class ClockFactoryTest { - - @Test - public void testDefaultIsSystemClock() { - System.clearProperty(ClockFactory.PROPERTY_NAME); - assertEquals(SystemClock.class, ClockFactory.getClock().getClass()); - } - - @Test - public void testSpecifySystemClockShort() { - System.setProperty(ClockFactory.PROPERTY_NAME, "SystemClock"); - assertEquals(SystemClock.class, ClockFactory.getClock().getClass()); - } - - @Test - public void testSpecifySystemClockLong() { - System.setProperty(ClockFactory.PROPERTY_NAME, SystemClock.class.getName()); - assertEquals(SystemClock.class, ClockFactory.getClock().getClass()); - } - - @Test - public void testSpecifyCachedClockShort() { - System.setProperty(ClockFactory.PROPERTY_NAME, "CachedClock"); - assertEquals(CachedClock.class, ClockFactory.getClock().getClass()); - } - - @Test - public void testSpecifyCachedClockLong() { - System.setProperty(ClockFactory.PROPERTY_NAME, CachedClock.class.getName()); - assertEquals(CachedClock.class, ClockFactory.getClock().getClass()); - } - - @Test - public void testSpecifyCoarseCachedClockShort() { - System.setProperty(ClockFactory.PROPERTY_NAME, "CoarseCachedClock"); - assertEquals(CoarseCachedClock.class, ClockFactory.getClock().getClass()); - } - - @Test - public void testSpecifyCoarseCachedClockLong() { - System.setProperty(ClockFactory.PROPERTY_NAME, CoarseCachedClock.class.getName()); - assertEquals(CoarseCachedClock.class, ClockFactory.getClock().getClass()); - } - - static class MyClock implements Clock { - @Override - public long currentTimeMillis() { - return 42; - } - } - - @Test - public void testCustomClock() { - System.setProperty(ClockFactory.PROPERTY_NAME, MyClock.class.getName()); - assertEquals(MyClock.class, ClockFactory.getClock().getClass()); - } - -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/SystemClockTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/SystemClockTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/SystemClockTest.java (working copy) @@ -1,46 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async; - -import static org.junit.Assert.*; - -import org.junit.Test; - -public class SystemClockTest { - - @Test - public void testLessThan2Millis() { - long millis1 = new SystemClock().currentTimeMillis(); - long sysMillis = System.currentTimeMillis(); - - long diff = sysMillis - millis1; - - assertTrue("diff too large: " + diff, diff <= 1); - } - - @Test - public void testAfterWaitStillLessThan2Millis() throws Exception { - Thread.sleep(100); - long millis1 = new SystemClock().currentTimeMillis(); - long sysMillis = System.currentTimeMillis(); - - long diff = sysMillis - millis1; - - assertTrue("diff too large: " + diff, diff <= 1); - } - -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastFileAppenderLocationTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastFileAppenderLocationTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastFileAppenderLocationTest.java (working copy) @@ -1,60 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.appender; - -import static org.junit.Assert.*; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; - -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.LifeCycle; -import org.apache.logging.log4j.core.config.XMLConfigurationFactory; -import org.junit.BeforeClass; -import org.junit.Test; - -public class FastFileAppenderLocationTest { - - @BeforeClass - public static void beforeClass() { - System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, - "FastFileAppenderLocationTest.xml"); - } - - @Test - public void testLocationIncluded() throws Exception { - File f = new File("target", "FastFileAppenderLocationTest.log"); - // System.out.println(f.getAbsolutePath()); - f.delete(); - Logger log = LogManager.getLogger("com.foo.Bar"); - String msg = "Message with location, flushed with immediate flush=false"; - log.info(msg); - ((LifeCycle) LogManager.getContext()).stop(); // stop async thread - - BufferedReader reader = new BufferedReader(new FileReader(f)); - String line1 = reader.readLine(); - reader.close(); - f.delete(); - assertNotNull("line1", line1); - assertTrue("line1 correct", line1.contains(msg)); - - String location = "testLocationIncluded"; - assertTrue("has location", line1.contains(location)); - } -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastFileAppenderTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastFileAppenderTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastFileAppenderTest.java (working copy) @@ -1,60 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.appender; - -import static org.junit.Assert.*; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; - -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.LifeCycle; -import org.apache.logging.log4j.core.config.XMLConfigurationFactory; -import org.junit.BeforeClass; -import org.junit.Test; - -public class FastFileAppenderTest { - - @BeforeClass - public static void beforeClass() { - System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, - "FastFileAppenderTest.xml"); - } - - @Test - public void testFlushAtEndOfBatch() throws Exception { - File f = new File("target", "FastFileAppenderTest.log"); - // System.out.println(f.getAbsolutePath()); - f.delete(); - Logger log = LogManager.getLogger("com.foo.Bar"); - String msg = "Message flushed with immediate flush=false"; - log.info(msg); - ((LifeCycle) LogManager.getContext()).stop(); // stop async thread - - BufferedReader reader = new BufferedReader(new FileReader(f)); - String line1 = reader.readLine(); - reader.close(); - f.delete(); - assertNotNull("line1", line1); - assertTrue("line1 correct", line1.contains(msg)); - - String location = "testFlushAtEndOfBatch"; - assertTrue("no location", !line1.contains(location)); - } -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastRollingFileAppenderLocationTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastRollingFileAppenderLocationTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastRollingFileAppenderLocationTest.java (working copy) @@ -1,60 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.appender; - -import static org.junit.Assert.*; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; - -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.LifeCycle; -import org.apache.logging.log4j.core.config.XMLConfigurationFactory; -import org.junit.BeforeClass; -import org.junit.Test; - -public class FastRollingFileAppenderLocationTest { - - @BeforeClass - public static void beforeClass() { - System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, - "FastRollingFileAppenderLocationTest.xml"); - } - - @Test - public void testLocationIncluded() throws Exception { - File f = new File("target", "FastRollingFileAppenderLocationTest.log"); - // System.out.println(f.getAbsolutePath()); - f.delete(); - Logger log = LogManager.getLogger("com.foo.Bar"); - String msg = "Message with location, flushed with immediate flush=false"; - log.info(msg); - ((LifeCycle) LogManager.getContext()).stop(); // stop async thread - - BufferedReader reader = new BufferedReader(new FileReader(f)); - String line1 = reader.readLine(); - reader.close(); - f.delete(); - assertNotNull("line1", line1); - assertTrue("line1 correct", line1.contains(msg)); - - String location = "testLocationIncluded"; - assertTrue("has location", line1.contains(location)); - } -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastRollingFileAppenderRolloverTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastRollingFileAppenderRolloverTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastRollingFileAppenderRolloverTest.java (working copy) @@ -1,92 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.appender; - -import static org.junit.Assert.*; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; -import java.util.Arrays; - -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.LifeCycle; -import org.apache.logging.log4j.core.config.XMLConfigurationFactory; -import org.junit.BeforeClass; -import org.junit.Test; - -public class FastRollingFileAppenderRolloverTest { - - @BeforeClass - public static void beforeClass() { - System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, - "FastRollingFileAppenderTest.xml"); - } - - @Test - public void testRollover() throws Exception { - File f = new File("target", "FastRollingFileAppenderTest.log"); - // System.out.println(f.getAbsolutePath()); - File after1 = new File("target", "afterRollover-1.log"); - f.delete(); - after1.delete(); - - Logger log = LogManager.getLogger("com.foo.Bar"); - String msg = "First a short message that does not trigger rollover"; - log.info(msg); - Thread.sleep(50); - - BufferedReader reader = new BufferedReader(new FileReader(f)); - String line1 = reader.readLine(); - assertTrue(line1.contains(msg)); - reader.close(); - - assertFalse("afterRollover-1.log not created yet", after1.exists()); - - String exceed = "Long message that exceeds rollover size... "; - char[] padding = new char[250]; - Arrays.fill(padding, 'X'); - exceed += new String(padding); - log.warn(exceed); - assertFalse("exceeded size but afterRollover-1.log not created yet", after1.exists()); - - String trigger = "This message triggers rollover."; - log.warn(trigger); - - ((LifeCycle) LogManager.getContext()).stop(); // stop async thread - - assertTrue("afterRollover-1.log created", after1.exists()); - - reader = new BufferedReader(new FileReader(f)); - String new1 = reader.readLine(); - assertTrue("after rollover only new msg", new1.contains(trigger)); - assertNull("No more lines", reader.readLine()); - reader.close(); - f.delete(); - - reader = new BufferedReader(new FileReader(after1)); - String old1 = reader.readLine(); - assertTrue("renamed file line 1", old1.contains(msg)); - String old2 = reader.readLine(); - assertTrue("renamed file line 2", old2.contains(exceed)); - String line = reader.readLine(); - assertNull("No more lines", line); - reader.close(); - after1.delete(); - } -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastRollingFileAppenderTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastRollingFileAppenderTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/appender/FastRollingFileAppenderTest.java (working copy) @@ -1,60 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.appender; - -import static org.junit.Assert.*; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; - -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.LifeCycle; -import org.apache.logging.log4j.core.config.XMLConfigurationFactory; -import org.junit.BeforeClass; -import org.junit.Test; - -public class FastRollingFileAppenderTest { - - @BeforeClass - public static void beforeClass() { - System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, - "FastRollingFileAppenderTest.xml"); - } - - @Test - public void testFlushAtEndOfBatch() throws Exception { - File f = new File("target", "FastRollingFileAppenderTest.log"); - // System.out.println(f.getAbsolutePath()); - f.delete(); - Logger log = LogManager.getLogger("com.foo.Bar"); - String msg = "Message flushed with immediate flush=false"; - log.info(msg); - ((LifeCycle) LogManager.getContext()).stop(); // stop async thread - - BufferedReader reader = new BufferedReader(new FileReader(f)); - String line1 = reader.readLine(); - reader.close(); - f.delete(); - assertNotNull("line1", line1); - assertTrue("line1 correct", line1.contains(msg)); - - String location = "testFlushAtEndOfBatch"; - assertTrue("no location", !line1.contains(location)); - } -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/config/AsyncLoggerConfigTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/config/AsyncLoggerConfigTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/config/AsyncLoggerConfigTest.java (working copy) @@ -1,65 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache license, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the license for the specific language governing permissions and - * limitations under the license. - */ -package org.apache.logging.log4j.async.config; - -import static org.junit.Assert.*; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; - -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.LifeCycle; -import org.apache.logging.log4j.core.config.XMLConfigurationFactory; -import org.junit.BeforeClass; -import org.junit.Test; - -public class AsyncLoggerConfigTest { - - @BeforeClass - public static void beforeClass() { - System.setProperty(XMLConfigurationFactory.CONFIGURATION_FILE_PROPERTY, - "AsyncLoggerConfigTest.xml"); - } - - @Test - public void testAdditivity() throws Exception { - File f = new File("target", "AsyncLoggerConfigTest.log"); - // System.out.println(f.getAbsolutePath()); - f.delete(); - Logger log = LogManager.getLogger("com.foo.Bar"); - String msg = "Additive logging: 2 for the price of 1!"; - log.info(msg); - ((LifeCycle) LogManager.getContext()).stop(); // stop async thread - - BufferedReader reader = new BufferedReader(new FileReader(f)); - String line1 = reader.readLine(); - String line2 = reader.readLine(); - reader.close(); - f.delete(); - assertNotNull("line1", line1); - assertNotNull("line2", line2); - assertTrue("line1 correct", line1.contains(msg)); - assertTrue("line2 correct", line2.contains(msg)); - - String location = "testAdditivity"; - assertTrue("location", - line1.contains(location) || line2.contains(location)); - } - -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/IPerfTestRunner.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/IPerfTestRunner.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/IPerfTestRunner.java (working copy) @@ -1,17 +0,0 @@ -package org.apache.logging.log4j.async.perftest; - -import com.lmax.disruptor.collections.Histogram; - -public interface IPerfTestRunner { - static final String LINE100 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890!\"#$%&'()-=^~|\\@`[]{};:+*,.<>/?_123456"; - static final String THROUGHPUT_MSG = LINE100 + LINE100 + LINE100 + LINE100 - + LINE100; - static final String LATENCY_MSG = "Short msg"; - - void runThroughputTest(int lines, Histogram histogram); - - void runLatencyTest(int samples, Histogram histogram, long nanoTimeCost, - int threadCount); - void shutdown(); - void log(String finalMessage); -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/MTPerfTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/MTPerfTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/MTPerfTest.java (working copy) @@ -1,93 +0,0 @@ -package org.apache.logging.log4j.async.perftest; - -import java.io.File; - -import com.lmax.disruptor.collections.Histogram; - -public class MTPerfTest extends PerfTest { - - public static void main(String[] args) throws Exception { - new MTPerfTest().doMain(args); - } - - @Override - public void runTestAndPrintResult(final IPerfTestRunner runner, - final String name, final int threadCount, String resultFile) - throws Exception { - - // ThreadContext.put("aKey", "mdcVal"); - PerfTest.println("Warming up the JVM..."); - long t1 = System.nanoTime(); - - // warmup at least 2 rounds and at most 1 minute - final Histogram warmupHist = PerfTest.createHistogram(); - final long stop = System.currentTimeMillis() + (60 * 1000); - Runnable run1 = new Runnable() { - public void run() { - for (int i = 0; i < 10; i++) { - final int LINES = PerfTest.throughput ? 50000 : 200000; - runTest(runner, LINES, null, warmupHist, 2); - if (i > 0 && System.currentTimeMillis() >= stop) { - return; - } - } - } - }; - Thread thread1 = new Thread(run1); - Thread thread2 = new Thread(run1); - thread1.start(); - thread2.start(); - thread1.join(); - thread2.join(); - - PerfTest.printf("Warmup complete in %.1f seconds%n", - (System.nanoTime() - t1) / (1000.0 * 1000.0 * 1000.0)); - PerfTest.println("Waiting 10 seconds for buffers to drain warmup data..."); - Thread.sleep(10000); - new File("perftest.log").delete(); - new File("perftest.log").createNewFile(); - - PerfTest.println("Starting the main test..."); - PerfTest.throughput = false; - multiThreadedTestRun(runner, name, threadCount, resultFile); - - Thread.sleep(1000); - PerfTest.throughput = true; - multiThreadedTestRun(runner, name, threadCount, resultFile); - } - - private void multiThreadedTestRun(final IPerfTestRunner runner, - final String name, final int threadCount, String resultFile) - throws Exception { - - final Histogram[] histograms = new Histogram[threadCount]; - for (int i = 0; i < histograms.length; i++) { - histograms[i] = PerfTest.createHistogram(); - } - final int LINES = 256 * 1024; - - Thread[] threads = new Thread[threadCount]; - for (int i = 0; i < threads.length; i++) { - final Histogram histogram = histograms[i]; - threads[i] = new Thread() { - public void run() { -// int latencyCount = threadCount >= 16 ? 1000000 : 5000000; - int latencyCount = 5000000; - int count = PerfTest.throughput ? LINES / threadCount - : latencyCount; - runTest(runner, count, "end", histogram, threadCount); - } - }; - } - for (Thread thread : threads) { - thread.start(); - } - for (Thread thread : threads) { - thread.join(); - } - - for (Histogram histogram : histograms) { - PerfTest.reportResult(resultFile, name, histogram); - } - } -} \ No newline at end of file Index: log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/PerfTest.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/PerfTest.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/PerfTest.java (working copy) @@ -1,168 +0,0 @@ -package org.apache.logging.log4j.async.perftest; - -import java.io.FileWriter; -import java.io.IOException; - -import com.lmax.disruptor.collections.Histogram; - -public class PerfTest { - - private static final String LINE100 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890!\"#$%&'()-=^~|\\@`[]{};:+*,.<>/?_123456"; - public static final String LINE500 = LINE100 + LINE100 + LINE100 + LINE100 - + LINE100; - - static boolean verbose = false; - static boolean throughput; - - // determine how long it takes to call System.nanoTime() (on average) - static long calcNanoTimeCost() { - final long iterations = 10000000; - long start = System.nanoTime(); - long finish = start; - - for (int i = 0; i < iterations; i++) { - finish = System.nanoTime(); - } - - if (finish <= start) { - throw new IllegalStateException(); - } - - finish = System.nanoTime(); - return (finish - start) / iterations; - } - - static Histogram createHistogram() { - long[] intervals = new long[31]; - long intervalUpperBound = 1L; - for (int i = 0, size = intervals.length - 1; i < size; i++) { - intervalUpperBound *= 2; - intervals[i] = intervalUpperBound; - } - - intervals[intervals.length - 1] = Long.MAX_VALUE; - return new Histogram(intervals); - } - - public static void main(String[] args) throws Exception { - new PerfTest().doMain(args); - } - - public void doMain(String[] args) throws Exception { - String runnerClass = args[0]; - IPerfTestRunner runner = (IPerfTestRunner) Class.forName(runnerClass) - .newInstance(); - String name = args[1]; - String resultFile = args.length > 2 ? args[2] : null; - for (String arg : args) { - if ("-verbose".equalsIgnoreCase(arg)) { - verbose = true; - } - if ("-throughput".equalsIgnoreCase(arg)) { - throughput = true; - } - } - int threadCount = args.length > 2 ? Integer.parseInt(args[3]) : 3; - printf("Starting %s %s (%d)...%n", getClass().getSimpleName(), name, - threadCount); - runTestAndPrintResult(runner, name, threadCount, resultFile); - runner.shutdown(); - System.exit(0); - } - - public void runTestAndPrintResult(IPerfTestRunner runner, - final String name, int threadCount, String resultFile) - throws Exception { - Histogram warmupHist = createHistogram(); - - // ThreadContext.put("aKey", "mdcVal"); - println("Warming up the JVM..."); - long t1 = System.nanoTime(); - - // warmup at least 2 rounds and at most 1 minute - final long stop = System.currentTimeMillis() + (60 * 1000); - for (int i = 0; i < 10; i++) { - final int LINES = throughput ? 50000 : 200000; - runTest(runner, LINES, null, warmupHist, 1); - if (i > 0 && System.currentTimeMillis() >= stop) { - return; - } - } - - printf("Warmup complete in %.1f seconds%n", (System.nanoTime() - t1) - / (1000.0 * 1000.0 * 1000.0)); - println("Waiting 10 seconds for buffers to drain warmup data..."); - Thread.sleep(10000); - - println("Starting the main test..."); - // test - throughput = false; - runSingleThreadedTest(runner, name, resultFile); - - Thread.sleep(1000); - - throughput = true; - runSingleThreadedTest(runner, name, resultFile); - } - - private int runSingleThreadedTest(IPerfTestRunner runner, String name, - String resultFile) throws IOException { - Histogram latency = createHistogram(); - final int LINES = throughput ? 50000 : 5000000; - runTest(runner, LINES, "end", latency, 1); - reportResult(resultFile, name, latency); - return LINES; - } - - static void reportResult(String file, String name, Histogram histogram) - throws IOException { - String result = createSamplingReport(name, histogram); - println(result); - - if (file != null) { - FileWriter writer = new FileWriter(file, true); - writer.write(result); - writer.write(System.getProperty("line.separator")); - writer.close(); - } - } - - static void printf(String msg, Object... objects) { - if (verbose) { - System.out.printf(msg, objects); - } - } - - static void println(String msg) { - if (verbose) { - System.out.println(msg); - } - } - - static String createSamplingReport(String name, Histogram histogram) { - Histogram data = histogram; - if (throughput) { - return data.getMax() + " operations/second"; - } - String result = String.format( - "avg=%.0f 99%%=%d 99.99%%=%d sampleCount=%d", data.getMean(), // - data.getTwoNinesUpperBound(), // - data.getFourNinesUpperBound(), // - data.getCount() // - ); - return result; - } - - public void runTest(IPerfTestRunner runner, int lines, String finalMessage, - Histogram histogram, int threadCount) { - if (throughput) { - runner.runThroughputTest(lines, histogram); - } else { - long nanoTimeCost = calcNanoTimeCost(); - runner.runLatencyTest(lines, histogram, nanoTimeCost, threadCount); - } - if (finalMessage != null) { - runner.log(finalMessage); - } - } -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/PerfTestDriver.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/PerfTestDriver.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/PerfTestDriver.java (working copy) @@ -1,347 +0,0 @@ -package org.apache.logging.log4j.async.perftest; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileReader; -import java.io.IOException; -import java.io.InputStreamReader; -import java.nio.CharBuffer; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -public class PerfTestDriver { - static enum WaitStrategy { - Sleep, Yield, Block - }; - - static class Setup implements Comparable { - private Class _class; - private String _log4jConfig; - private String _name; - private String[] _systemProperties; - private int _threadCount; - private File _temp; - public Stats _stats; - private WaitStrategy _wait; - private String _runner; - - public Setup(Class klass, String runner, String name, - String log4jConfig, int threadCount, WaitStrategy wait, - String... systemProperties) throws IOException { - _class = klass; - _runner = runner; - _name = name; - _log4jConfig = log4jConfig; - _threadCount = threadCount; - _systemProperties = systemProperties; - _wait = wait; - _temp = File.createTempFile("log4jperformance", ".txt"); - } - - List processArguments(String java) { - List args = new ArrayList(); - args.add(java); - args.add("-server"); - args.add("-Xms1g"); - args.add("-Xmx1g"); - - // args.add("-XX:+UseParallelOldGC"); - // args.add("-Xloggc:gc.log"); - // args.add("-XX:+PrintGCTimeStamps"); - // args.add("-XX:+PrintGCDetails"); - // args.add("-XX:+PrintGCDateStamps"); - // args.add("-XX:+PrintGCApplicationStoppedTime"); - // args.add("-XX:+PrintGCApplicationConcurrentTime"); - // args.add("-XX:+PrintSafepointStatistics"); - - args.add("-Dlog4j.configuration=" + _log4jConfig); // log4j1.2 - args.add("-Dlog4j.configurationFile=" + _log4jConfig); // log4j2 - args.add("-Dlogback.configurationFile=" + _log4jConfig);// logback - - int ringBufferSize = getUserSpecifiedRingBufferSize(); - if (ringBufferSize >= 128) { - args.add("-DAsyncLoggerConfig.RingBufferSize=" + ringBufferSize); - args.add("-DAsyncLogger.RingBufferSize=" + ringBufferSize); - } - args.add("-DAsyncLoggerConfig.WaitStrategy=" + _wait); - args.add("-DAsyncLogger.WaitStrategy=" + _wait); - if (_systemProperties != null) { - for (String property : _systemProperties) { - args.add(property); - } - } - args.add("-cp"); - args.add(System.getProperty("java.class.path")); - args.add(_class.getName()); - args.add(_runner); - args.add(_name); - args.add(_temp.getAbsolutePath()); - args.add(String.valueOf(_threadCount)); - return args; - } - - private int getUserSpecifiedRingBufferSize() { - try { - return Integer.parseInt(System.getProperty("RingBufferSize", - "-1")); - } catch (Exception ignored) { - return -1; - } - } - - ProcessBuilder latencyTest(String java) { - return new ProcessBuilder(processArguments(java)); - } - - ProcessBuilder throughputTest(String java) { - List args = processArguments(java); - args.add("-throughput"); - return new ProcessBuilder(args); - } - - @Override - public int compareTo(Setup other) { - // largest ops/sec first - return (int) Math.signum(other._stats._averageOpsPerSec - - _stats._averageOpsPerSec); - } - - public String description() { - String detail = _class.getSimpleName(); - if (PerfTest.class == _class) { - detail = "single thread"; - } else if (MTPerfTest.class == _class) { - detail = _threadCount + " threads"; - } - String target = _runner.substring(_runner.indexOf(".Run") + 4); - return target + ": " + _name + " (" + detail + ")"; - } - } - - static class Stats { - int _count; - long _average; - long _pct99; - long _pct99_99; - double _latencyRowCount; - int _throughputRowCount; - private long _averageOpsPerSec; - - // example line: avg=828 99%=1118 99.99%=5028 Count=3125 - public Stats(String raw, int repeat) { - String[] lines = raw.split("[\\r\\n]+"); - long totalOps = 0; - for (String line : lines) { - if (line.startsWith("avg")) { - _latencyRowCount++; - String[] parts = line.split(" "); - int i = 0; - _average += Long.parseLong(parts[i++].split("=")[1]); - _pct99 += Long.parseLong(parts[i++].split("=")[1]); - _pct99_99 += Long.parseLong(parts[i++].split("=")[1]); - _count += Integer.parseInt(parts[i++].split("=")[1]); - } else { - _throughputRowCount++; - String number = line.substring(0, line.indexOf(' ')); - long opsPerSec = Long.parseLong(number); - totalOps += opsPerSec; - } - } - _averageOpsPerSec = totalOps / (int) _throughputRowCount; - } - - public String toString() { - String fmt = "throughput: %,d ops/sec. latency(ns): avg=%.1f 99%% < %.1f 99.99%% < %.1f (%d samples)"; - return String.format(fmt, _averageOpsPerSec, // - _average / _latencyRowCount, // mean latency - _pct99 / _latencyRowCount, // 99% observations less than - _pct99_99 / _latencyRowCount,// 99.99% observs less than - _count); - } - } - - // single-threaded performance test - private static Setup s(String config, String runner, String name, - String... systemProperties) throws IOException { - WaitStrategy wait = WaitStrategy.valueOf(System.getProperty( - "WaitStrategy", "Sleep")); - return new Setup(PerfTest.class, runner, name, config, 1, wait, - systemProperties); - } - - // multi-threaded performance test - private static Setup m(String config, String runner, String name, - int threadCount, String... systemProperties) throws IOException { - WaitStrategy wait = WaitStrategy.valueOf(System.getProperty( - "WaitStrategy", "Sleep")); - return new Setup(MTPerfTest.class, runner, name, config, threadCount, - wait, systemProperties); - } - - public static void main(String[] args) throws Exception { - final String ALL_ASYNC = "-DLog4jContextSelector=org.apache.logging.log4j.async.AsyncLoggerContextSelector"; - final String CACHEDCLOCK = "-DAsyncLogger.Clock=CachedClock"; - final String SYSCLOCK = ""; - final String LOG12 = "org.apache.logging.log4j.async.perftest.RunLog4j1"; - final String LOG20 = "org.apache.logging.log4j.async.perftest.RunLog4j2"; - final String LOGBK = "org.apache.logging.log4j.async.perftest.RunLogback"; - - long start = System.nanoTime(); - List tests = new ArrayList(); - // includeLocation=false - tests.add(s("perf-logback.xml", LOGBK, "Sync")); - tests.add(s("perf-log4j12.xml", LOG12, "Sync")); - tests.add(s("perf3PlainNoLoc.xml", LOG20, "Sync")); - tests.add(s("perf3PlainNoLoc.xml", LOG20, "Loggers all async", - ALL_ASYNC, SYSCLOCK)); - tests.add(s("perf7MixedNoLoc.xml", LOG20, "Loggers mixed sync/async")); - tests.add(s("perf-logback-async.xml", LOGBK, "Async Appender")); - tests.add(s("perf-log4j12-async.xml", LOG12, "Async Appender")); - tests.add(s("perf5AsyncApndNoLoc.xml", LOG20, "Async Appender")); - - // includeLocation=true - // tests.add(s("perf6AsyncApndLoc.xml", LOG20, - // "Async Appender includeLocation")); - // tests.add(s("perf8MixedLoc.xml", LOG20, - // "Mixed sync/async includeLocation")); - // tests.add(s("perf4PlainLocation.xml", LOG20, - // "Loggers all async includeLocation", ALL_ASYNC)); - // tests.add(s("perf4PlainLocation.xml", LOG20, - // "Loggers all async includeLocation CachedClock", ALL_ASYNC, - // CACHEDCLOCK)); - // tests.add(s("perf4PlainLocation.xml", LOG20, - // "Sync includeLocation")); - - // appenders - // tests.add(s("perf1syncFile.xml", LOG20, "FileAppender")); - // tests.add(s("perf1syncFastFile.xml", LOG20, "FastFileAppender")); - // tests.add(s("perf2syncRollFile.xml", LOG20, "RollFileAppender")); - // tests.add(s("perf2syncRollFastFile.xml", LOG20, - // "RollFastFileAppender")); - - final int MAX_THREADS = 16; // 64 takes a LONG time - for (int i = 2; i <= MAX_THREADS; i *= 2) { - // includeLocation = false - tests.add(m("perf-logback.xml", LOGBK, "Sync", i)); - tests.add(m("perf-log4j12.xml", LOG12, "Sync", i)); - tests.add(m("perf3PlainNoLoc.xml", LOG20, "Sync", i)); - tests.add(m("perf-logback-async.xml", LOGBK, "Async Appender", i)); - tests.add(m("perf-log4j12-async.xml", LOG12, "Async Appender", i)); - tests.add(m("perf5AsyncApndNoLoc.xml", LOG20, "Async Appender", i)); - tests.add(m("perf3PlainNoLoc.xml", LOG20, "Loggers all async", i, - ALL_ASYNC, SYSCLOCK)); - tests.add(m("perf7MixedNoLoc.xml", LOG20, - "Loggers mixed sync/async", i)); - - // includeLocation=true - // tests.add(m("perf6AsyncApndLoc.xml", LOG20, - // "Async Appender includeLocation", i)); - // tests.add(m("perf8MixedLoc.xml", LOG20, - // "Mixed sync/async includeLocation", i)); - // tests.add(m("perf4PlainLocation.xml", LOG20, - // "Loggers all async includeLocation", i, ALL_ASYNC)); - // tests.add(m("perf4PlainLocation.xml", LOG20, - // "Loggers all async includeLocation CachedClock", i, - // ALL_ASYNC, CACHEDCLOCK)); - // tests.add(m("perf4PlainLocation.xml", LOG20, - // "Sync includeLocation", i)); - - // appenders - // tests.add(m("perf1syncFile.xml", LOG20, "FileAppender", i)); - // tests.add(m("perf1syncFastFile.xml", LOG20, "FastFileAppender", - // i)); - // tests.add(m("perf2syncRollFile.xml", LOG20, "RollFileAppender", - // i)); - // tests.add(m("perf2syncRollFastFile.xml", LOG20, - // "RollFastFileAppender", i)); - } - - String java = args.length > 0 ? args[0] : "java"; - int repeat = args.length > 1 ? Integer.parseInt(args[1]) : 5; - int x = 0; - for (Setup config : tests) { - System.out.print(config.description()); - ProcessBuilder pb = config.throughputTest(java); - pb.redirectErrorStream(true); // merge System.out and System.err - long t1 = System.nanoTime(); - // int count = config._threadCount >= 16 ? 2 : repeat; - int count = repeat; - runPerfTest(count, x++, config, pb); - System.out.printf(" took %.1f seconds%n", (System.nanoTime() - t1) - / (1000.0 * 1000.0 * 1000.0)); - - FileReader reader = new FileReader(config._temp); - CharBuffer buffer = CharBuffer.allocate(256 * 1024); - reader.read(buffer); - reader.close(); - config._temp.delete(); - buffer.flip(); - - String raw = buffer.toString(); - System.out.print(raw); - Stats stats = new Stats(raw, repeat); - System.out.println(stats); - System.out.println("-----"); - config._stats = stats; - } - new File("perftest.log").delete(); - System.out - .printf("Done. Total duration: %.1f minutes%n", - (System.nanoTime() - start) - / (60.0 * 1000.0 * 1000.0 * 1000.0)); - - printRanking((Setup[]) tests.toArray(new Setup[tests.size()])); - } - - private static void printRanking(Setup[] tests) { - System.out.println(); - System.out.println("Ranking:"); - Arrays.sort(tests); - for (int i = 0; i < tests.length; i++) { - Setup setup = tests[i]; - System.out.println((i + 1) + ". " + setup.description() + ": " - + setup._stats); - } - } - - private static void runPerfTest(int repeat, int setupIndex, Setup config, - ProcessBuilder pb) throws IOException, InterruptedException { - for (int i = 0; i < repeat; i++) { - System.out.print(" (" + (i + 1) + "/" + repeat + ")..."); - final Process process = pb.start(); - - final boolean[] stop = { false }; - printProcessOutput(process, stop); - process.waitFor(); - stop[0] = true; - - File gc = new File("gc" + setupIndex + "_" + i - + config._log4jConfig + ".log"); - if (gc.exists()) { - gc.delete(); - } - new File("gc.log").renameTo(gc); - } - } - - private static Thread printProcessOutput(final Process process, - final boolean[] stop) { - - Thread t = new Thread("OutputWriter") { - public void run() { - BufferedReader in = new BufferedReader(new InputStreamReader( - process.getInputStream())); - try { - String line = null; - while (!stop[0] && (line = in.readLine()) != null) { - System.out.println(line); - } - } catch (Exception ignored) { - } - } - }; - t.start(); - return t; - } -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/PerfTestResultFormatter.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/PerfTestResultFormatter.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/PerfTestResultFormatter.java (working copy) @@ -1,167 +0,0 @@ -package org.apache.logging.log4j.async.perftest; - -import java.io.BufferedReader; -import java.io.InputStreamReader; -import java.text.DecimalFormat; -import java.text.NumberFormat; -import java.text.ParseException; -import java.util.Arrays; -import java.util.Comparator; -import java.util.List; -import java.util.Map; -import java.util.Set; -import java.util.TreeMap; - -/** - * Utility class that can read the "Ranking" output of the PerfTestDriver and - * format it for pasting into Excel. - */ -class PerfTestResultFormatter { - static final String LF = System.getProperty("line.separator"); - static final NumberFormat NUM = new DecimalFormat("#,##0"); - - static class Stats { - long throughput; - double avgLatency; - double latency99Pct; - double latency99_99Pct; - - Stats(String throughput, String avg, String lat99, String lat99_99) - throws ParseException { - this.throughput = NUM.parse(throughput.trim()).longValue(); - this.avgLatency = Double.parseDouble(avg.trim()); - this.latency99Pct = Double.parseDouble(lat99.trim()); - this.latency99_99Pct = Double.parseDouble(lat99_99.trim()); - } - } - - private Map> results = new TreeMap>(); - - public PerfTestResultFormatter() { - } - - public String format(String text) throws ParseException { - results.clear(); - String[] lines = text.split("[\\r\\n]+"); - for (String line : lines) { - process(line); - } - return latencyTable() + LF + throughputTable(); - } - - private String latencyTable() { - StringBuilder sb = new StringBuilder(4 * 1024); - Set subKeys = results.values().iterator().next().keySet(); - char[] tabs = new char[subKeys.size()]; - Arrays.fill(tabs, '\t'); - String sep = new String(tabs); - sb.append("\tAverage latency" + sep + "99% less than" + sep - + "99.99% less than"); - sb.append(LF); - for (int i = 0; i < 3; i++) { - for (String subKey : subKeys) { - sb.append("\t").append(subKey); - } - } - sb.append(LF); - for (String key : results.keySet()) { - sb.append(key); - for (int i = 0; i < 3; i++) { - Map sub = results.get(key); - for (String subKey : sub.keySet()) { - Stats stats = sub.get(subKey); - switch (i) { - case 0: - sb.append("\t").append((long) stats.avgLatency); - break; - case 1: - sb.append("\t").append((long) stats.latency99Pct); - break; - case 2: - sb.append("\t").append((long) stats.latency99_99Pct); - break; - } - } - } - sb.append(LF); - } - return sb.toString(); - } - - private String throughputTable() { - StringBuilder sb = new StringBuilder(4 * 1024); - Set subKeys = results.values().iterator().next().keySet(); - sb.append("\tThroughput per thread (msg/sec)"); - sb.append(LF); - for (String subKey : subKeys) { - sb.append("\t").append(subKey); - } - sb.append(LF); - for (String key : results.keySet()) { - sb.append(key); - Map sub = results.get(key); - for (String subKey : sub.keySet()) { - Stats stats = sub.get(subKey); - sb.append("\t").append((long) stats.throughput); - } - sb.append(LF); - } - return sb.toString(); - } - - private void process(String line) throws ParseException { - String key = line.substring(line.indexOf('.') + 1, line.indexOf('(')); - String sub = line.substring(line.indexOf('(') + 1, line.indexOf(')')); - String throughput = line.substring(line.indexOf("throughput: ") - + "throughput: ".length(), line.indexOf(" ops")); - String avg = line.substring(line.indexOf("avg=") + "avg=".length(), - line.indexOf(" 99%")); - String pct99 = line.substring( - line.indexOf("99% < ") + "99% < ".length(), - line.indexOf(" 99.99%")); - String pct99_99 = line.substring(line.indexOf("99.99% < ") - + "99.99% < ".length(), line.lastIndexOf('(') - 1); - Stats stats = new Stats(throughput, avg, pct99, pct99_99); - Map map = results.get(key.trim()); - if (map == null) { - map = new TreeMap(sort()); - results.put(key.trim(), map); - } - String subKey = sub.trim(); - if ("single thread".equals(subKey)) { - subKey = "1 thread"; - } - map.put(subKey, stats); - } - - private Comparator sort() { - return new Comparator() { - List expected = Arrays.asList("1 thread", "2 threads", - "4 threads", "8 threads", "16 threads", "32 threads", - "64 threads"); - - @Override - public int compare(String o1, String o2) { - int i1 = expected.indexOf(o1); - int i2 = expected.indexOf(o2); - if (i1 < 0 || i2 < 0) { - return o1.compareTo(o2); - } - return i1 - i2; - } - }; - } - - public static void main(String[] args) throws Exception { - PerfTestResultFormatter fmt = new PerfTestResultFormatter(); - BufferedReader reader = new BufferedReader(new InputStreamReader( - System.in)); - String line = null; - while ((line = reader.readLine()) != null) { - fmt.process(line); - } - System.out.println(fmt.latencyTable()); - System.out.println(); - System.out.println(fmt.throughputTable()); - } -} \ No newline at end of file Index: log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/RunLog4j1.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/RunLog4j1.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/RunLog4j1.java (working copy) @@ -1,53 +0,0 @@ -package org.apache.logging.log4j.async.perftest; - -import org.apache.log4j.LogManager; -import org.apache.log4j.Logger; - -import com.lmax.disruptor.collections.Histogram; - -public class RunLog4j1 implements IPerfTestRunner { - - @Override - public void runThroughputTest(int lines, Histogram histogram) { - long s1 = System.nanoTime(); - Logger logger = LogManager.getLogger(getClass()); - for (int j = 0; j < lines; j++) { - logger.info(THROUGHPUT_MSG); - } - long s2 = System.nanoTime(); - long opsPerSec = (1000L * 1000L * 1000L * lines) / (s2 - s1); - histogram.addObservation(opsPerSec); - } - - @Override - public void runLatencyTest(int samples, Histogram histogram, - long nanoTimeCost, int threadCount) { - Logger logger = LogManager.getLogger(getClass()); - for (int i = 0; i < samples; i++) { - long s1 = System.nanoTime(); - logger.info(LATENCY_MSG); - long s2 = System.nanoTime(); - long value = s2 - s1 - nanoTimeCost; - if (value > 0) { - histogram.addObservation(value); - } - // wait 1 microsec - final long PAUSE_NANOS = 10000 * threadCount; - long pauseStart = System.nanoTime(); - while (PAUSE_NANOS > (System.nanoTime() - pauseStart)) { - // busy spin - } - } - } - - @Override - public void shutdown() { - LogManager.shutdown(); - } - - @Override - public void log(String finalMessage) { - Logger logger = LogManager.getLogger(getClass()); - logger.info(finalMessage); - } -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/RunLog4j2.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/RunLog4j2.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/RunLog4j2.java (working copy) @@ -1,57 +0,0 @@ -package org.apache.logging.log4j.async.perftest; - -import org.apache.logging.log4j.LogManager; -import org.apache.logging.log4j.Logger; -import org.apache.logging.log4j.core.LifeCycle; - -import com.lmax.disruptor.collections.Histogram; - -public class RunLog4j2 implements IPerfTestRunner { - - @Override - public void runThroughputTest(int lines, Histogram histogram) { - long s1 = System.nanoTime(); - Logger logger = LogManager.getLogger(getClass()); - for (int j = 0; j < lines; j++) { - logger.info(THROUGHPUT_MSG); - } - long s2 = System.nanoTime(); - long opsPerSec = (1000L * 1000L * 1000L * lines) / (s2 - s1); - histogram.addObservation(opsPerSec); - } - - - @Override - public void runLatencyTest(int samples, Histogram histogram, - long nanoTimeCost, int threadCount) { - Logger logger = LogManager.getLogger(getClass()); - for (int i = 0; i < samples; i++) { - long s1 = System.nanoTime(); - logger.info(LATENCY_MSG); - long s2 = System.nanoTime(); - long value = s2 - s1 - nanoTimeCost; - if (value > 0) { - histogram.addObservation(value); - } - // wait 1 microsec - final long PAUSE_NANOS = 10000 * threadCount; - long pauseStart = System.nanoTime(); - while (PAUSE_NANOS > (System.nanoTime() - pauseStart)) { - // busy spin - } - } - } - - - @Override - public void shutdown() { - ((LifeCycle) LogManager.getContext()).stop(); // stop async thread - } - - - @Override - public void log(String finalMessage) { - Logger logger = LogManager.getLogger(getClass()); - logger.info(finalMessage); - } -} Index: log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/RunLogback.java =================================================================== --- log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/RunLogback.java (revision 1467676) +++ log4j-async/src/test/java/org/apache/logging/log4j/async/perftest/RunLogback.java (working copy) @@ -1,55 +0,0 @@ -package org.apache.logging.log4j.async.perftest; - -import org.slf4j.LoggerFactory; - -import ch.qos.logback.classic.Logger; -import ch.qos.logback.core.spi.LifeCycle; - -import com.lmax.disruptor.collections.Histogram; - -public class RunLogback implements IPerfTestRunner { - - @Override - public void runThroughputTest(int lines, Histogram histogram) { - long s1 = System.nanoTime(); - Logger logger = (Logger) LoggerFactory.getLogger(getClass()); - for (int j = 0; j < lines; j++) { - logger.info(THROUGHPUT_MSG); - } - long s2 = System.nanoTime(); - long opsPerSec = (1000L * 1000L * 1000L * lines) / (s2 - s1); - histogram.addObservation(opsPerSec); - } - - @Override - public void runLatencyTest(int samples, Histogram histogram, - long nanoTimeCost, int threadCount) { - Logger logger = (Logger) LoggerFactory.getLogger(getClass()); - for (int i = 0; i < samples; i++) { - long s1 = System.nanoTime(); - logger.info(LATENCY_MSG); - long s2 = System.nanoTime(); - long value = s2 - s1 - nanoTimeCost; - if (value > 0) { - histogram.addObservation(value); - } - // wait 1 microsec - final long PAUSE_NANOS = 10000 * threadCount; - long pauseStart = System.nanoTime(); - while (PAUSE_NANOS > (System.nanoTime() - pauseStart)) { - // busy spin - } - } - } - - @Override - public void shutdown() { - ((LifeCycle) LoggerFactory.getILoggerFactory()).stop(); - } - - @Override - public void log(String msg) { - Logger logger = (Logger) LoggerFactory.getLogger(getClass()); - logger.info(msg); - } -} Index: log4j-async/src/test/resources/AsyncLoggerConfigTest.xml =================================================================== --- log4j-async/src/test/resources/AsyncLoggerConfigTest.xml (revision 1467676) +++ log4j-async/src/test/resources/AsyncLoggerConfigTest.xml (working copy) @@ -1,19 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %location %ex%n - - - - - - - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/AsyncLoggerLocationTest.xml =================================================================== --- log4j-async/src/test/resources/AsyncLoggerLocationTest.xml (revision 1467676) +++ log4j-async/src/test/resources/AsyncLoggerLocationTest.xml (working copy) @@ -1,17 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %location %m %ex%n - - - - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/AsyncLoggerTest.xml =================================================================== --- log4j-async/src/test/resources/AsyncLoggerTest.xml (revision 1467676) +++ log4j-async/src/test/resources/AsyncLoggerTest.xml (working copy) @@ -1,17 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %location %m %ex%n - - - - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/FastFileAppenderLocationTest.xml =================================================================== --- log4j-async/src/test/resources/FastFileAppenderLocationTest.xml (revision 1467676) +++ log4j-async/src/test/resources/FastFileAppenderLocationTest.xml (working copy) @@ -1,19 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %location %ex%n - - - - - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/FastFileAppenderTest.xml =================================================================== --- log4j-async/src/test/resources/FastFileAppenderTest.xml (revision 1467676) +++ log4j-async/src/test/resources/FastFileAppenderTest.xml (working copy) @@ -1,19 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %location %ex%n - - - - - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/FastRollingFileAppenderLocationTest.xml =================================================================== --- log4j-async/src/test/resources/FastRollingFileAppenderLocationTest.xml (revision 1467676) +++ log4j-async/src/test/resources/FastRollingFileAppenderLocationTest.xml (working copy) @@ -1,21 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %location %ex%n - - - - - - - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/FastRollingFileAppenderTest.xml =================================================================== --- log4j-async/src/test/resources/FastRollingFileAppenderTest.xml (revision 1467676) +++ log4j-async/src/test/resources/FastRollingFileAppenderTest.xml (working copy) @@ -1,21 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %location %ex%n - - - - - - - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/log4j.dtd =================================================================== --- log4j-async/src/test/resources/log4j.dtd (revision 1467676) +++ log4j-async/src/test/resources/log4j.dtd (working copy) @@ -1,170 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Index: log4j-async/src/test/resources/perf-log4j12-async.xml =================================================================== --- log4j-async/src/test/resources/perf-log4j12-async.xml (revision 1467676) +++ log4j-async/src/test/resources/perf-log4j12-async.xml (working copy) @@ -1,20 +0,0 @@ - - - - - - - - - - - - - - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/perf-log4j12.xml =================================================================== --- log4j-async/src/test/resources/perf-log4j12.xml (revision 1467676) +++ log4j-async/src/test/resources/perf-log4j12.xml (working copy) @@ -1,16 +0,0 @@ - - - - - - - - - - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/perf-logback-async.xml =================================================================== --- log4j-async/src/test/resources/perf-logback-async.xml (revision 1467676) +++ log4j-async/src/test/resources/perf-logback-async.xml (working copy) @@ -1,21 +0,0 @@ - - - - perftest.log - false - - %d %p %c{1} [%t] %X{aKey} %m %ex%n - false - - - - 262144 - 0 - false - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/perf-logback.xml =================================================================== --- log4j-async/src/test/resources/perf-logback.xml (revision 1467676) +++ log4j-async/src/test/resources/perf-logback.xml (working copy) @@ -1,15 +0,0 @@ - - - - perftest.log - false - - %d %p %c{1} [%t] %X{aKey} %m %ex%n - false - - - - - - - \ No newline at end of file Index: log4j-async/src/test/resources/perf1syncFastFile.xml =================================================================== --- log4j-async/src/test/resources/perf1syncFastFile.xml (revision 1467676) +++ log4j-async/src/test/resources/perf1syncFastFile.xml (working copy) @@ -1,15 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %ex%n - - - - - - - - - Index: log4j-async/src/test/resources/perf1syncFile.xml =================================================================== --- log4j-async/src/test/resources/perf1syncFile.xml (revision 1467676) +++ log4j-async/src/test/resources/perf1syncFile.xml (working copy) @@ -1,15 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %ex%n - - - - - - - - - Index: log4j-async/src/test/resources/perf2syncRollFastFile.xml =================================================================== --- log4j-async/src/test/resources/perf2syncRollFastFile.xml (revision 1467676) +++ log4j-async/src/test/resources/perf2syncRollFastFile.xml (working copy) @@ -1,20 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %ex%n - - - - - - - - - - - - Index: log4j-async/src/test/resources/perf2syncRollFile.xml =================================================================== --- log4j-async/src/test/resources/perf2syncRollFile.xml (revision 1467676) +++ log4j-async/src/test/resources/perf2syncRollFile.xml (working copy) @@ -1,20 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %ex%n - - - - - - - - - - - - Index: log4j-async/src/test/resources/perf3PlainNoLoc.xml =================================================================== --- log4j-async/src/test/resources/perf3PlainNoLoc.xml (revision 1467676) +++ log4j-async/src/test/resources/perf3PlainNoLoc.xml (working copy) @@ -1,15 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %ex%n - - - - - - - - - Index: log4j-async/src/test/resources/perf4PlainLocation.xml =================================================================== --- log4j-async/src/test/resources/perf4PlainLocation.xml (revision 1467676) +++ log4j-async/src/test/resources/perf4PlainLocation.xml (working copy) @@ -1,15 +0,0 @@ - - - - - - %d %p %c{1.} %C %location %line [%t] %X{aKey} %m %ex%n - - - - - - - - - Index: log4j-async/src/test/resources/perf5AsyncApndNoLoc.xml =================================================================== --- log4j-async/src/test/resources/perf5AsyncApndNoLoc.xml (revision 1467676) +++ log4j-async/src/test/resources/perf5AsyncApndNoLoc.xml (working copy) @@ -1,18 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %ex%n - - - - - - - - - - - - Index: log4j-async/src/test/resources/perf6AsyncApndLoc.xml =================================================================== --- log4j-async/src/test/resources/perf6AsyncApndLoc.xml (revision 1467676) +++ log4j-async/src/test/resources/perf6AsyncApndLoc.xml (working copy) @@ -1,18 +0,0 @@ - - - - - - %d %p %c{1.} %C %location %line [%t] %X{aKey} %m %ex%n - - - - - - - - - - - - Index: log4j-async/src/test/resources/perf7MixedNoLoc.xml =================================================================== --- log4j-async/src/test/resources/perf7MixedNoLoc.xml (revision 1467676) +++ log4j-async/src/test/resources/perf7MixedNoLoc.xml (working copy) @@ -1,15 +0,0 @@ - - - - - - %d %p %c{1.} [%t] %X{aKey} %m %ex%n - - - - - - - - - Index: log4j-async/src/test/resources/perf8MixedLoc.xml =================================================================== --- log4j-async/src/test/resources/perf8MixedLoc.xml (revision 1467676) +++ log4j-async/src/test/resources/perf8MixedLoc.xml (working copy) @@ -1,15 +0,0 @@ - - - - - - %d %p %c{1.} %C %location %line [%t] %X{aKey} %m %ex%n - - - - - - - - - \ No newline at end of file Index: src/site/pdf.xml =================================================================== --- src/site/pdf.xml (revision 1467676) +++ src/site/pdf.xml (working copy) @@ -37,6 +37,7 @@ + Index: src/site/site.xml =================================================================== --- src/site/site.xml (revision 1467676) +++ src/site/site.xml (working copy) @@ -73,6 +73,8 @@ + + @@ -101,13 +103,11 @@ - + - - @@ -132,7 +132,6 @@ - Index: src/site/xdoc/index.xml =================================================================== --- src/site/xdoc/index.xml (revision 1467676) +++ src/site/xdoc/index.xml (working copy) @@ -44,7 +44,14 @@
Improved Performance
- Log4j 2 performs faster than Log4j 1.x in critical areas and similarly to Logback under most circumstances. + Log4j 2 contains next-generation Asynchronous Loggers based + on the LMAX Disruptor library. In multi-threaded scenarios + Asynchronous Loggers have 10 times higher throughput and + orders of magnitude lower latency than Log4j 1.x and Logback. + See Asynchronous Logging Performance + for details. + Otherwise, Log4j 2 performs faster than Log4j 1.x in critical areas + and similarly to Logback under most circumstances. See Performance for more information.
Support for multiple APIs
Index: src/site/xdoc/manual/appenders.xml =================================================================== --- src/site/xdoc/manual/appenders.xml (revision 1467676) +++ src/site/xdoc/manual/appenders.xml (working copy) @@ -1354,6 +1354,380 @@ ]]>

+ + +

Experimental, may replace FileAppender in a future release.

+

+ The FastFileAppender is similar to the standard + FileAppender + except it is always buffered (this cannot be switched off) + and internally it uses a + ByteBuffer + RandomAccessFile + instead of a + BufferedOutputStream. + We saw a 20-200% performance improvement compared to + FileAppender with "bufferedIO=true" in our + measurements. + Similar to the FileAppender, + FastFileAppender uses a FastFileManager to actually perform the + file I/O. While FastFileAppender + from different Configurations + cannot be shared, the FastFileManagers can be if the Manager is + accessible. For example, two webapps in a + servlet container can have + their own configuration and safely + write to the same file if Log4j + is in a ClassLoader that is common to + both of them. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Parameter NameTypeDescription
appendbooleanWhen true - the default, records will be appended to the end + of the file. When set to false, + the file will be cleared before + new records are written. +
fileNameStringThe name of the file to write to. If the file, or any of its + parent directories, do not exist, + they will be created. +
filtersFilterA Filter to determine if the event should be handled by this + Appender. More than one Filter + may be used by using a CompositeFilter. +
immediateFlushbooleanWhen set to true, each write will be followed by a flush. + This will guarantee the data is written + to disk but could impact performance. + This option is only useful when using this + appender with synchronous loggers. Asynchronous loggers will + automatically flush at the end of a batch, which also guarantees + the data is written to disk but is more efficient. +
layoutLayoutThe Layout to use to format the LogEvent
nameStringThe name of the Appender.
suppressExceptionsbooleanThe default is true, causing exceptions to be internally + logged and then ignored. When set to + false exceptions will be + percolated to the caller. +
FastFileAppender Parameters
+

+ Here is a sample FastFile configuration: + +


+
+  
+    
+      
+        %d %p %c{1.} [%t] %m%n
+      
+    
+  
+  
+    
+      
+    
+  
+]]>
+

+ + + +

Experimental, may replace RollingFileAppender in a future release.

+

+ The FastRollingFileAppender is similar to the standard + RollingFileAppender + except it is always buffered (this cannot be switched off) + and + internally it uses a + ByteBuffer + RandomAccessFile + instead of a + BufferedOutputStream. + We saw a 20-200% performance improvement compared to + RollingFileAppender with "bufferedIO=true" + in our + measurements. + + The FastRollingFileAppender writes + to the File named in the + fileName parameter + and rolls the file over according the + TriggeringPolicy + and the RolloverPolicy. + + Similar to the RollingFileAppender, + FastRollingFileAppender uses a FastRollingFileManager + to actually perform the + file I/O and perform the rollover. While FastRollingFileAppender + from different Configurations cannot be + shared, the FastRollingFileManagers can be + if the Manager is accessible. + For example, two webapps in a servlet + container can have their own configuration and safely write to the + same file if Log4j is in a ClassLoader that is common to both of them. +

+

+ A FastRollingFileAppender requires a + TriggeringPolicy + and a + RolloverStrategy. + The triggering policy determines if a rollover should + be performed + while the RolloverStrategy defines how the rollover + should be done. + If no RolloverStrategy + is configured, FastRollingFileAppender will + use the + DefaultRolloverStrategy. +

+

+ File locking is not supported by the FastRollingFileAppender. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Parameter NameTypeDescription
appendbooleanWhen true - the default, records will be appended to the end + of the file. When set to false, + the file will be cleared before + new records are written. +
filterFilterA Filter to determine if the event should be handled by this + Appender. More than one Filter + may be used by using a + CompositeFilter. +
fileNameStringThe name of the file to write to. If the file, or any of its + parent directories, do not exist, + they will be created. +
filePatternString + The pattern of the file name of the archived log file. The format + of the pattern should is + dependent on the RolloverPolicy that is + used. The DefaultRolloverPolicy + will accept both + a date/time + pattern compatible with + + SimpleDateFormat + + and/or a %i which represents an integer counter. The pattern + also supports interpolation at + runtime so any of the Lookups (such + as the + DateLookup + can + be included in the pattern. +
immediateFlushbooleanWhen set to true, each write will be followed by a flush. This + will guarantee the data is written + to disk but could impact + performance. + This option is only useful when using this + appender with + synchronous loggers. Asynchronous loggers will + automatically + flush at the end of a batch, which also guarantees + the data + is written to disk but is more efficient. +
layoutLayoutThe Layout to use to format the LogEvent
nameStringThe name of the Appender.
policyTriggeringPolicyThe policy to use to determine if a rollover should occur. +
strategyRolloverStrategyThe strategy to use to determine the name and location of the + archive file. +
suppressExceptionsbooleanThe default is true, causing exceptions to be internally + logged and then ignored. When set to + false exceptions will be + percolated to the caller. +
FastRollingFileAppender Parameters
+ +

Triggering Policies

+

+ See + RollingFileAppender Triggering Policies. +

+ +

Rollover Strategies

+

+ See + RollingFileAppender Rollover Strategies. +

+ +

+ Below is a sample configuration that uses a FastRollingFileAppender + with both the time and size based + triggering policies, will create + up to 7 archives on the same day (1-7) that + are stored in a + directory + based on the current year and month, and will compress + each + archive using gzip: + +


+
+  
+    
+      
+        %d %p %c{1.} [%t] %m%n
+      
+      
+        
+        
+      
+    
+  
+  
+    
+      
+    
+  
+]]>
+

+

+ This second example shows a rollover strategy that will keep up to + 20 files before removing them. +


+
+  
+    
+      
+        %d %p %c{1.} [%t] %m%n
+      
+      
+        
+        
+      
+      
+    
+  
+  
+    
+      
+    
+  
+]]>
+

+

+ Below is a sample configuration that uses a FastRollingFileAppender + with both the time and size based + triggering policies, will create + up to 7 archives on the same day (1-7) that + are stored in a + directory + based on the current year and month, and will compress + each + archive using gzip and will roll every 6 hours when the hour is + divisible + by 6: + +


+
+  
+    
+      
+        %d %p %c{1.} [%t] %m%n
+      
+      
+        
+        
+      
+    
+  
+  
+    
+      
+    
+  
+]]>
+

+

Index: src/site/xdoc/manual/async.xml =================================================================== --- src/site/xdoc/manual/async.xml (revision 1467676) +++ src/site/xdoc/manual/async.xml (working copy) @@ -11,11 +11,11 @@ language governing permissions and limitations under the License. --> - Log4j2 Asynchronous Logging + Log4j2 Asynchronous Loggers for Low-Latency Logging Remko Popma -

+

Asynchronous logging can improve your application's performance by executing the I/O operations @@ -53,7 +53,7 @@ Appenders. Under the hood, these new appenders use a ByteBuffer + RandomAccessFile instead of a BufferedOutputStream. In our testing - this was about 10-30% faster. + this was about 20-200% faster. These appenders can also be used with synchronous loggers and will give the same performance benefits. @@ -125,11 +125,11 @@

This is simplest to configure and gives the best performance. - To make all loggers asynchronous, add log4j-async.jar - and disruptor-3.0.0.jar to the classpath + To make all loggers asynchronous, + add disruptor-3.0.0.jar to the classpath and set the system property Log4jContextSelector to - org.apache.logging.log4j.async.AsyncLoggerContextSelector. + org.apache.logging.log4j.core.async.AsyncLoggerContextSelector.

By default, location is not passed to the I/O thread by @@ -270,7 +270,7 @@ Implementation of the - org.apache.logging.log4j.async.Clock + org.apache.logging.log4j.core.async.Clock interface that is used for timestamping the log events when all loggers are asynchronous.
@@ -302,6 +302,7 @@ +

Requires disruptor-3.0.0.jar on the classpath.

Synchronous and asynchronous loggers can be combined in configuration. This gives you more flexibility at the cost @@ -345,7 +346,8 @@ - + + @@ -469,378 +471,6 @@

-
- -

- The FastFileAppender is similar to the standard - FileAppender - except it is always buffered (this cannot be switched off) - and internally it uses a - ByteBuffer + RandomAccessFile - instead of a - BufferedOutputStream. - We saw a 20-200% performance improvement compared to - FileAppender with "bufferedIO=true" in our - measurements. - Similar to the FileAppender, - FastFileAppender uses a FastFileManager to actually perform the - file I/O. While FastFileAppender - from different Configurations - cannot be shared, the FastFileManagers can be if the Manager is - accessible. For example, two webapps in a - servlet container can have - their own configuration and safely - write to the same file if Log4j - is in a ClassLoader that is common to - both of them. -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Parameter NameTypeDescription
appendbooleanWhen true - the default, records will be appended to the end - of the file. When set to false, - the file will be cleared before - new records are written. -
fileNameStringThe name of the file to write to. If the file, or any of its - parent directories, do not exist, - they will be created. -
filtersFilterA Filter to determine if the event should be handled by this - Appender. More than one Filter - may be used by using a CompositeFilter. -
immediateFlushbooleanWhen set to true, each write will be followed by a flush. - This will guarantee the data is written - to disk but could impact performance. - This option is only necessary when using this - appender with synchronous loggers. Asynchronous loggers will - automatically flush at the end of a batch, which also guarantees - the data is written to disk but is more efficient. -
layoutLayoutThe Layout to use to format the LogEvent
nameStringThe name of the Appender.
suppressExceptionsbooleanThe default is true, causing exceptions to be internally - logged and then ignored. When set to - false exceptions will be - percolated to the caller. -
FastFileAppender Parameters
-

- Here is a sample FastFile configuration: - -


-
-  
-    
-      
-        %d %p %c{1.} [%t] %m%n
-      
-    
-  
-  
-    
-      
-    
-  
-]]>
-

- - - -

- The FastRollingFileAppender is similar to the standard - RollingFileAppender - except it is always buffered (this cannot be switched off) - and - internally it uses a - ByteBuffer + RandomAccessFile - instead of a - BufferedOutputStream. - We saw a 20-200% performance improvement compared to - RollingFileAppender with "bufferedIO=true" - in our - measurements. - - The FastRollingFileAppender writes - to the File named in the - fileName parameter - and rolls the file over according the - TriggeringPolicy - and the RolloverPolicy. - - Similar to the RollingFileAppender, - FastRollingFileAppender uses a FastRollingFileManager - to actually perform the - file I/O and perform the rollover. While FastRollingFileAppender - from different Configurations cannot be - shared, the FastRollingFileManagers can be - if the Manager is accessible. - For example, two webapps in a servlet - container can have their own configuration and safely write to the - same file if Log4j is in a ClassLoader that is common to both of them. -

-

- A FastRollingFileAppender requires a - TriggeringPolicy - and a - RolloverStrategy. - The triggering policy determines if a rollover should - be performed - while the RolloverStrategy defines how the rollover - should be done. - If no RolloverStrategy - is configured, FastRollingFileAppender will - use the - DefaultRolloverStrategy. -

-

- File locking is not supported by the FastRollingFileAppender. -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Parameter NameTypeDescription
appendbooleanWhen true - the default, records will be appended to the end - of the file. When set to false, - the file will be cleared before - new records are written. -
filterFilterA Filter to determine if the event should be handled by this - Appender. More than one Filter - may be used by using a - CompositeFilter. -
fileNameStringThe name of the file to write to. If the file, or any of its - parent directories, do not exist, - they will be created. -
filePatternString - The pattern of the file name of the archived log file. The format - of the pattern should is - dependent on the RolloverPolicy that is - used. The DefaultRolloverPolicy - will accept both - a date/time - pattern compatible with - - SimpleDateFormat - - and/or a %i which represents an integer counter. The pattern - also supports interpolation at - runtime so any of the Lookups (such - as the - DateLookup - can - be included in the pattern. -
immediateFlushbooleanWhen set to true, each write will be followed by a flush. This - will guarantee the data is written - to disk but could impact - performance. - This option is only necessary when using this - appender with - synchronous loggers. Asynchronous loggers will - automatically - flush at the end of a batch, which also guarantees - the data - is written to disk but is more efficient. -
layoutLayoutThe Layout to use to format the LogEvent
nameStringThe name of the Appender.
policyTriggeringPolicyThe policy to use to determine if a rollover should occur. -
strategyRolloverStrategyThe strategy to use to determine the name and location of the - archive file. -
suppressExceptionsbooleanThe default is true, causing exceptions to be internally - logged and then ignored. When set to - false exceptions will be - percolated to the caller. -
FastRollingFileAppender Parameters
- -

Triggering Policies

-

- See - RollingFileAppender Triggering Policies. -

- -

Rollover Strategies

-

- See - RollingFileAppender Rollover Strategies. -

- -

- Below is a sample configuration that uses a FastRollingFileAppender - with both the time and size based - triggering policies, will create - up to 7 archives on the same day (1-7) that - are stored in a - directory - based on the current year and month, and will compress - each - archive using gzip: - -


-
-  
-    
-      
-        %d %p %c{1.} [%t] %m%n
-      
-      
-        
-        
-      
-    
-  
-  
-    
-      
-    
-  
-]]>
-

-

- This second example shows a rollover strategy that will keep up to - 20 files before removing them. -


-
-  
-    
-      
-        %d %p %c{1.} [%t] %m%n
-      
-      
-        
-        
-      
-      
-    
-  
-  
-    
-      
-    
-  
-]]>
-

-

- Below is a sample configuration that uses a FastRollingFileAppender - with both the time and size based - triggering policies, will create - up to 7 archives on the same day (1-7) that - are stored in a - directory - based on the current year and month, and will compress - each - archive using gzip and will roll every 6 hours when the hour is - divisible - by 6: - -


-
-  
-    
-      
-        %d %p %c{1.} [%t] %m%n
-      
-      
-        
-        
-      
-    
-  
-  
-    
-      
-    
-  
-]]>
-

-

@@ -1141,7 +771,8 @@

- The scale of the latency comparison graphs is logarithmic, not linear. + Note that the scale of the latency comparison graphs is + logarithmic, not linear. That makes it a bit difficult to see, but the average latency of asynchronous loggers is half to one third of the latency of ArrayBlockingQueue-based asynchronous appenders in scenarios @@ -1155,10 +786,12 @@ The graph below shows that asynchronous loggers also do better when comparing the maximum latency of 99.99% of observations with other logging methods. - The worst case latency of asynchronous loggers remains more or - less the same (around 10-20 microseconds) in multi-threaded scenarios where - ArrayBlockingQueue-based asynchronous appenders have more outliers - with latencies of over 100 milliseconds per call to Logger.log. + When increasing the number of threads + the worst case latency of asynchronous loggers remains more or + less the same (around 10-20 microseconds) + where Asynchronous Appenders start experiencing worst-case + latency spikes in the 100 millisecond range, a difference of + four orders of magnitude.

Index: src/site/xdoc/manual/index.xml =================================================================== --- src/site/xdoc/manual/index.xml (revision 1467676) +++ src/site/xdoc/manual/index.xml (working copy) @@ -88,11 +88,13 @@ Logback exceptions in Appenders are never visible to the application. In Log4j 2 Appenders can be configured to allow the exception to percolate to the application +

  • Log4j 2 contains next-generation Asynchronous Loggers based + on the LMAX Disruptor library. In multi-threaded scenarios + Asynchronous Loggers have 10 times higher throughput and + orders of magnitude lower latency than Log4j 1.x and Logback.
  • Log4j 2 uses a Plugin system that makes it extremely easy to extend the framework by adding new Appenders, Filters, Layouts, Lookups, and Pattern Converters without requiring any changes to Log4j.
  • -
  • The performance of Log4j 2 is similar to that of Logback. It is slightly - slower in some tests and faster in others.
  • Due to the Plugin system configuration is simpler. Entries in the configuration do not require a class name to be specified.
  • Support for Message objects. Messages allow support for interesting and