Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.13.0
-
None
-
Unknown
Description
We do use the file component to read files from a particular area. We pick up new files from there using their content as basis for a database import. There are a couple of important points:
- We must not add or delete files to the area we read from, not even temporarily.
- We need to pick up files once.
To achive this we use noop=true as well as a file based idempotentRepository where the repository is placed in a different location. We also use readLock=changed which is the big problem. To cite the component documentation:
Notice from Camel 2.10 onwards the read locks changed, fileLock and rename will also use a markerFile as well, to ensure not picking up files that may be in process by another Camel consumer running on another node (eg cluster).
This is a huge problem as I can't find any documented way of storing the marker files in a different location. What is really needed is something similar to inProgressRepository as an option to keep track of the readLock related information. Alternatively an option to disable the marker files could be created, this option could however lead to failure if applications are clustered.
This problem is for us serious breakage, as there are other applications not under our control reading the same areas, some of these crashes due to the marker files.