[FLUME-432] Add ability to provide directory filter to tailDir sink in case of recursing into subdirs

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 0.9.4
• Fix Version/s: 0.9.5
• Component/s: Sinks+Sources
• Assignee: Unassigned
• Reporter: Alex Baranau
• Created: 27/Dec/10 09:07
• Updated: 05/Nov/14 09:07
• Resolved: 05/Nov/14 09:07

Description

Currently tailDir accepts regex parameter that is used to filter files to be tailed. In case of recursing into subdirs (FLUME-410) filter is not applied to directories. Ability to filter directories to recurse into (i.e. the ones to tail) should be provided.

Suggestions:

• add extra param for filtering dirs on regex
• filter files & dirs based on full path. E.g. user could use "subdir[12](/..log)?" to tail subdir1/.log and subdir2/*.log

Issue Links

• is blocked by: FLUME-410 Add ability to tail subdirs to tailDir sink