Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
0.9.3
Description
It appears that having a line with length of 32767 bytes or more
throws the TailSource into a, um, tail spin. I've set
flume.event.max.size.bytes to 1000000, so that shouldn't be the
limitation. Simple steps to reproduce:
flume-0.9.3rc1> ruby -e 'puts "x"*32767' > /tmp/log
flume-0.9.3rc1> bin/flume dump 'tail("/tmp/log")'
It appears that the source sends an incomplete message to the sink,
and then continuously retries to send it.
I've attached a patch that creates a buffer of size flume.event.max.size.bytes,
and if it detects a line longer than this, it gets thrown away. It's been working
for our systems for a week, but I haven't had time to write a unit test and test
the edge cases.