[FLUME-2911] Add includePattern option in SpoolDirectorySource configuration

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.0, 1.7.0
• Fix Version/s: 1.7.0
• Component/s: Sinks+Sources
• Labels: features
• Release Note: Added includePattern option to spooling directory source.
• Flags: Patch
• Assignee: Andrea Rota
• Reporter: Andrea Rota
• Created: 21/May/16 15:06
• Updated: 10/Oct/16 17:05
• Resolved: 10/Oct/16 15:32

Description

Current implementation of SpoolDirectorySource does not allow users to specify a regex pattern to select which files should be monitored. Instead, the current implementation allows users to specify which should not monitored, via the ignorePattern parameter.

I implemented the feature, allowing users to specify the include pattern as a1.sources.src-1.includePattern=^foo.*$ (includes all the files that starts in "foo").

By default, the includePattern regex is set to ^.*$ (all files). Include and exclude patterns can be used at same time and the results are combined.

Issue Links

• links to: GitHub Pull Request #60