[FLUME-1365] Flume-ng with syslog events break with certain characters in message body

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.1.0
• Component/s: Sinks+Sources

Environment

[root@datapipe1 varoun.p]# rpm -qa | grep -i flume
flume-ng-1.1.0+120-1.cdh4.0.0.p0.14.el6.noarch
flume-ng-agent-1.1.0+120-1.cdh4.0.0.p0.14.el6.noarch
[root@datapipe1 varoun.p]# uname -a
Linux datapipe1.tools.us-east-1b.aws.talk.to 3.2.12-3.2.4.amzn1.x86_64 #1 SMP Thu Mar 22 08:00:08 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
[root@datapipe1 varoun.p]#

• Assignee: Ashish Paliwal
• Reporter: Varoun P
• Created: 11/Jul/12 13:20
• Updated: 02/Dec/13 14:11

Description

When syslog messages contain hyphens separated by whitespace, flume fails to parse and create valid events. For example, I send three the messages to syslog:

[root@proxy1 varoun.p]# logger -i -p local1.notice 'foo bar' [root@proxy1 varoun.p]# logger -i -p local1.notice 'foo – bar'
[root@proxy1 varoun.p]# logger -i -p local1.notice 'foo - - bar'
[root@proxy1 varoun.p]#

The corresponding events created by flume are:

2012-07-11 13:17:17,558 INFO sink.LoggerSink: Event: { headers:

{timestamp=1342012635000, Severity=5, host=proxy1, Facility=136}

body: 76 61 72 6F 75 6E 2E 70 5B 31 33 32 38 35 5D 3A varoun.p[13285]: }
2012-07-11 13:17:19,264 INFO sink.LoggerSink: Event: { headers:

{timestamp=1342012639000, Severity=5, host=proxy1, Facility=136}

body: 76 61 72 6F 75 6E 2E 70 5B 31 33 32 38 36 5D 3A varoun.p[13286]: }
2012-07-11 13:17:23,265 INFO sink.LoggerSink: Event: { headers:

{Severity=5, Facility=136}

body: 62 61 72 bar }

As you can see, the last message results in a broken flume event.

Issue Links

• duplicates: FLUME-1277 Error parsing Syslog rfc 3164 messages with null values