[APEXMALHAR-1684] Improve file splitter not to emit all files in one shot and hold scanned file names in memory

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Component/s: adapters other
• Assignee: Chandni Singh
• Reporter: Chandni Singh
• Created: 27/Feb/15 19:28
• Updated: 03/Jan/16 10:01
• Resolved: 26/Mar/15 05:41

Description

Currently the file splitter scans all the files and emits them in one shot.
Also it holds the file paths in memory to know whether a file was discovered or not.

Changes:

• Added a time based directory scanner which will finds files which have been modified since the last scan.
• Added a blocks threshold property which can control no. of blocks emitted.