[FLUME-2282] File Channel throws IllegalStateException causing some sources to not be to recover from a reconfig

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Configuration, File Channel
• Labels: channel
• Assignee: Unassigned
• Reporter: Roshan Naik
• Created: 22/Dec/13 06:41
• Updated: 24/Dec/13 18:34

Description

when the config file is updated and the file channel is shut, the Source receives a java.lang.IllegalState exception from FileChannel. Unfortunately unless sources are designed to catch all exception types, they will not know this is a channel exception and likely to go into fuzzy states.

For instance the FLUME-2255 patch catches only ChannelException.

The below stack trace can be seen in logs when this happens with FLUME-2255 applied:

21 Dec 2013 21:15:03,030 ERROR [pool-17-thread-1] (org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run:194)  - Uncaught exception in SpoolDirSource thread 
java.lang.IllegalStateException: Channel closed [channel=channel_nontx]
	at org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:351)
	at org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)
	at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:181)
	at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:179)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:679)

Issue Links

• is broken by: FLUME-2283 Spool Dir source must check interrupt flag before writing to channel