[LOG4J2-1733] Add SyncSend attribute to KafkaAppender (as in KafkaLog4jAppender)

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.5
• Fix Version/s: 2.8
• Component/s: Appenders

Environment

Kafka 0.10.1.0

• External issue URL: http://stackoverflow.com/questions/41104429/poor-performance-of-log4j2-in-combination-with-kafka
• Assignee: Mikael Ståldal
• Reporter: Vincent Tieleman
• Created: 12/Dec/16 16:08
• Updated: 19/Apr/17 15:49
• Resolved: 23/Jan/17 16:17

Description

The KafkaLog4jAppender (shipped with Kafka and usable with log4j), has a syncsend property that improves performance significantly by not blocking a send to Kafka. I've tried many other configuration settings and setups, but find that none of these approaches improved the logging performance to Kafka as significantly as setting the syncsend property to false.

Unfortunately, the syncsend property is not supported by the KafkaAppender shipped with log4j2 and the KafkaLog4jAppender only supports log4j, so I am stuck with forking the code and making the change myself.

Could you please introduce this property in the KafkaAppender in log4j2?

Issue Links

• relates to: LOG4J2-1800 Errors when sending to Kafka with syncSend=false are not reported
• relates to: LOG4J2-1797 Add asynchronous/non-blocking SPIs for appenders
• requires: LOG4J2-1739 Add Builder to KafkaAppender and deprecate org.apache.logging.log4j.core.appender.mom.kafka.KafkaAppender.createAppender(Layout<? extends Serializable>, Filter, String, boolean, String, Property[], Configuration)
• links to: GitHub Pull Request #51