[FLUME-1934] Spoolingdir source exception when reading multiple zero size files

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.1
• Fix Version/s: 1.6.0

Environment

windows 7, flume 1.3.1, spooling dir source.

• Assignee: Grant Henke
• Reporter: andy zhou
• Created: 08/Mar/13 03:58
• Updated: 07/Apr/15 02:40
• Resolved: 07/Apr/15 01:16

Description

move more than one files to the spool dir, and each file size is 0, then flume agent will throw IllegalStateException forever, and never work again,its' main cause is commited flag will not set to true.

logs:

08 三月 2013 08:00:14,406 ERROR [pool-5-thread-1] (org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run:148) - Uncaught exception in Runnable
java.lang.IllegalStateException: File should not roll when commit is outstanding.
at org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:164)
at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:135)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

Issue Links

• duplicates: FLUME-2157 Spool directory source does not shut down correctly when Flume is reconfigured