Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.7.0
-
None
-
None
-
None
Description
Configured TailDirSource with multiple file groups. Some of the file group directories do not exist on all nodes on the cluster.
One the nodes where the file group directory did not exist TailDirSource failed to run with exception
05 Oct 2015 20:39:56,433 ERROR [lifecycleSupervisor-1-0] (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:253) - Unable to start PollableSourceRunner: { source:Taildir source: { positionFile: /dir/flume/taildir_position.json, skipToEnd: false, byteOffsetHeader: false, idleTimeout: 120000, writePosInterval: 3000 } counterGroup:{ name:null counters:{} } } - Exception follows. java.lang.IllegalStateException: Directory does not exist: /dir/log/hive at com.google.common.base.Preconditions.checkState(Preconditions.java:145) at org.apache.flume.source.taildir.ReliableTaildirEventReader.<init>(ReliableTaildirEventReader.java:86) at org.apache.flume.source.taildir.ReliableTaildirEventReader.<init>(ReliableTaildirEventReader.java:55) at org.apache.flume.source.taildir.ReliableTaildirEventReader$Builder.build(ReliableTaildirEventReader.java:343) at org.apache.flume.source.taildir.TaildirSource.start(TaildirSource.java:91) at org.apache.flume.source.PollableSourceRunner.start(PollableSourceRunner.java:74) at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
I built flume from sha: bd80c5e67deda4ec146f19fc92ff0c3bff61a982