[FLUME-2953] Make TaildirSource work with recursive directory

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.7.0
• Component/s: Sinks+Sources
• Labels: Recuresive TaildirSource Wildcards reviewboard-missing
• Assignee: Unassigned
• Reporter: tinawenqiao
• Created: 12/Jul/16 08:52
• Updated: 01/Jun/22 15:24

Description

In TaildirSource filegroupName, regular expression can be used for filename only. Sample usage is : a1.sources.r1.filegroups.f2 = /var/log/test2/.*log.*
If there are many files to be tracked in the same directory, the configuration is oft-repeated. So it‘s necessary that wildcards are supported in the directory path. Then the user can configure the filegroupName like this：
a1.sources.r1.filegroups.f2 = /var/log/*/.*log.*