Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
If a sink expects a header but does not find it, events will become stuck in the channel and Flume will log NullPointer and EventDelivery exceptions. In a memory channel, this can be fixed by restarting. In a file channel, restarting does not cause events to be removed.
05 Aug 2013 12:21:09,424 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:160) - Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: java.lang.NullPointerException: Expected timestamp in the Flume event headers, but it was null
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:426)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException: Expected timestamp in the Flume event headers, but it was null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:200)
at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:396)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:356)
... 3 more
05 Aug 2013 12:21:09,424 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.hdfs.HDFSEventSink.process:422) - process failed
java.lang.NullPointerException: Expected timestamp in the Flume event headers, but it was null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:200)
at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:396)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:356)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:662)
I was using RegexExtractorInterceptor to match timestamp for partitioning in with HDFS sink.
Attachments
Issue Links
- duplicates
-
FLUME-1264 HDFEventSink throws NPE if event is generated without timestamp in header
- Patch Available