Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.7.0
-
None
-
None
-
PRODUCTION
Description
We get the below exception in PROD.
08 Oct 2017 15:46:09,988 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:158) - Unable to deliver event. Exception follows.
org.apache.flume.ChannelException: Take failed due to IO error [channel=chn-file-xml-dis-log]
at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doTake(FileChannel.java:534)
at org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
at org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:95)
at com.macys.daas.flume.sink.elasticsearch.CreateIndex.process(CreateIndex.java:150)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Corrupt event found. Please run File Channel Integrity tool.
at org.apache.flume.channel.file.Log.get(Log.java:616)
at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doTake(FileChannel.java:531)
... 6 more
Caused by: org.apache.flume.channel.file.CorruptEventException: Could not parse event from data file.
at org.apache.flume.channel.file.TransactionEventRecord.fromByteArray(TransactionEventRecord.java:212)
at org.apache.flume.channel.file.LogFileV3$RandomReader.doGet(LogFileV3.java:303)
at org.apache.flume.channel.file.LogFile$RandomReader.get(LogFile.java:501)
at org.apache.flume.channel.file.Log.get(Log.java:612)
... 7 more
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message was too large. May be malicious. Use CodedInputStream.setSizeLimit() to increase the size limit.
at com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
at com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
at com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
at com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
at org.apache.flume.channel.file.proto.ProtosFactory$Put.<init>(ProtosFactory.java:3979)
at org.apache.flume.channel.file.proto.ProtosFactory$Put.<init>(ProtosFactory.java:3943)
at org.apache.flume.channel.file.proto.ProtosFactory$Put$1.parsePartialFrom(ProtosFactory.java:4039)
at org.apache.flume.channel.file.proto.ProtosFactory$Put$1.parsePartialFrom(ProtosFactory.java:4034)
at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
at com.google.protobuf.AbstractParser.parsePartialDelimitedFrom(AbstractParser.java:241)
at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:253)
at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:259)
at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:49)
at org.apache.flume.channel.file.proto.ProtosFactory$Put.parseDelimitedFrom(ProtosFactory.java:4179)
at org.apache.flume.channel.file.Put.readProtos(Put.java:97)
at org.apache.flume.channel.file.TransactionEventRecord.fromByteArray(TransactionEventRecord.java:206)
... 10 more
08 Oct 2017 15:46:14,866 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:158) - Unable to deliver event. Exception follows.
java.lang.IllegalStateException: Log is closed
at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.channel.file.Log.getFlumeEventQueue(Log.java:591)
at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.<init>(FileChannel.java:442)
at org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:359)
at org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)
at com.macys.daas.flume.sink.elasticsearch.CreateIndex.process(CreateIndex.java:136)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
at java.lang.Thread.run(Thread.java:748)