Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.5.0.1
-
None
-
None
Description
I am unable to send logs when I am using a FileChannel in Flume 1.5.0.1. Here is the stack trace that I am seeing:
java.lang.IllegalStateException: Channel closed [channel=channel]. Due to java.lang.UnsupportedOperationException: This is supposed to be overridden by subclasses.
at org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:352)
at org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)
at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:336)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.UnsupportedOperationException: This is supposed to be overridden by subclasses.
at com.google.protobuf.GeneratedMessage.getUnknownFields(GeneratedMessage.java:180)
at org.apache.flume.channel.file.proto.ProtosFactory$Checkpoint.getSerializedSize(ProtosFactory.java:233)
at com.google.protobuf.AbstractMessageLite.writeDelimitedTo(AbstractMessageLite.java:84)
at org.apache.flume.channel.file.EventQueueBackingStoreFileV3.<init>(EventQueueBackingStoreFileV3.java:111)
at org.apache.flume.channel.file.EventQueueBackingStoreFactory.get(EventQueueBackingStoreFactory.java:70)
at org.apache.flume.channel.file.Log.replay(Log.java:449)
at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:302)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
... 1 more
When I change the agent configuration to use the memory channel, the error goes away, and my logs get transferred successfully.
Source host configuration broken looks like this:
agent.channels = channel
agent.sinks = sink
agent.channels.channel.type = file
agent.channels.channel.checkpointDir = /media/ephemeral0/file-channel-test/checkpoint
agent.channels.channel.dataDirs = /media/ephemeral0/file-channel-test/data
agent.sinks.sink.channel = channel
agent.sinks.sink.type = avro
agent.sinks.sink.hostname = sink_hostname
agent.sinks.sink.port = 40000
agent.sources = source1
agent.sources.source1.type = exec
agent.sources.source1.command = tail -F /var/log/app.log
agent.sources.source1.channels = channel
However, using the same configuration, I am able to get the logs to successfully transfer when I change the configuration to use a memory channel instead:
By adding this:
agent.channels.channel.type = memory
And removing the file channel configuration:
agent.channels.channel.type = file
agent.channels.channel.checkpointDir = /media/ephemeral0/file-channel-test/checkpoint
agent.channels.channel.dataDirs = /media/ephemeral0/file-channel-test/data