[FLUME-410] Add ability to tail subdirs to tailDir sink

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.3
• Component/s: Sinks+Sources
• Assignee: Alex Baranau
• Reporter: Alex Baranau
• Created: 17/Dec/10 18:49
• Updated: 05/Feb/11 16:19
• Resolved: 28/Dec/10 20:55

Description

Add additional parameter to tailDir source to make it recurse into subdirectories for watching/tailing files. Ideally add parameter which defines max depth of recursing.

Motivation:

There are monitoring (and other type of tools) tools which write logs into multiple dirs, i.e. log files are not in a single dir but placed into multiple dirs with tree structure. E.g. collectd creates separate dir for each cpu and puts there logs.

Issue Links

• blocks: FLUME-431 Add attribute with dir path to events produced by tailing sources
• blocks: FLUME-432 Add ability to provide directory filter to tailDir sink in case of recursing into subdirs