[FLUME-383] E2E agent keeps dying when using batch and gzip decorators.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 0.9.2
• Fix Version/s: 0.9.5
• Component/s: Node

Environment

CentOS 5.4
Java build 1.6.0_13-b03

• Assignee: Unassigned
• Reporter: Disabled imported user
• Created: 10/Dec/10 17:57
• Updated: 05/Nov/14 08:58
• Resolved: 05/Nov/14 08:58

Description

When running in E2E using batch and gzip decorators, the agent dies.

Configs:
exec config 'xxx' 'syslogTcp(5140)' '{delay (500) => {batch(10000,30000) =>

{gzip => autoE2EChain}

}}'

exec config 'xxx' 'autoCollectorSource' '{gunzip =>

{unbatch => collectorSink("hdfs://dc1-had01-m001.dc01.revsci.net:9000/data/pixlogs","","60000")}

}'

Link to Flume user group thread
https://groups.google.com/a/cloudera.org/group/flume-user/browse_thread/thread/c3195f16324f43c8#

2010-12-06 15:29:57,669 ERROR
com.cloudera.flume.handlers.batch.BatchingDecorator: IOException when
ending batch!
java.io.IOException: java.lang.NullPointerException
at com.cloudera.flume.agent.durability.NaiveFileWALDeco
$1.fireError(NaiveFileWALDeco.java:250)
at
com.cloudera.flume.core.connector.DirectDriver.fireError(DirectDriver.java:
216)
at com.cloudera.flume.core.connector.DirectDriver
$PumperThread.run(DirectDriver.java:102)
Caused by: java.lang.NullPointerException
at
com.cloudera.flume.handlers.thrift.ThriftEventSink.append(ThriftEventSink.java:
76)
at
com.cloudera.flume.core.BackOffFailOverSink.append(BackOffFailOverSink.java:
144)
at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:
58)
at
com.cloudera.flume.handlers.debug.InsistentOpenDecorator.append(InsistentOpenDecorator.java:
163)
at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:
58)
at
com.cloudera.flume.handlers.debug.StubbornAppendSink.append(StubbornAppendSink.java:
84)
at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:
58)
at com.cloudera.flume.agent.durability.NaiveFileWALDeco
$AckChecksumRegisterer.append(NaiveFileWALDeco.java:99)
at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:
58)
at
com.cloudera.flume.handlers.debug.LazyOpenDecorator.append(LazyOpenDecorator.java:
70)
at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:
58)
at com.cloudera.flume.core.connector.DirectDriver
$PumperThread.run(DirectDriver.java:93)