[CAMEL-21056] Delay when using readLockMinAge

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.20.6
• Fix Version/s: 3.22.3
• Component/s: camel-file
• Estimated Complexity: Unknown
• Regression: Regression
• Assignee: Claus Ibsen
• Reporter: Stephan Balmer
• Created: 07/Aug/24 15:33
• Updated: 09/Aug/24 11:08
• Resolved: 08/Aug/24 09:40

Description

When using the file component with the readLockMinAge setting, consuming each file incurs a delay the length of readLockMinAge . So when `readLockMinAge=5s` and there are 1000 files, it takes 5000 seconds to consume all files.

Example route URL: file:/tmp/test?readLock=changed&readLockMinAge=5s&recursive=true&delete=true

This has two disadvantages:

• The maximal rate at which files are processed is 1/readLockMinAge. This severly limits throughput on a route.
• When a route is started it will initially wait for readLockMinAge before processing a file. Even if the file was written long ago.

Camel up to version 3.20.5 processes files instantly as soon as the min age is reached. Since version 3.20.6 files are not processed instantly even if they are older than min age. Instead, the file component will wait for readLockMinAge for each file individually.

This regression was introduced in commit https://github.com/apache/camel/commit/5459f1da2673205a6c24041a8631ac99088b5eae I can sympathize with the goal of that change, as I've experienced that problem as well because file-managers preserve last-changed dates when copying files to new places. However I do judge the importance of throughput in the usual case higher than avoiding premature processing of files in the edge case. I think the change should be reverted. A separate strategy or option for that use-case could be introduced.

Issue Links

• is caused by: CAMEL-19378 File Changed ReadLock Strategy with minAge only looks for lastModified