[CONNECTORS-13] We should augment file-based synchronization to allow a process/service-based synchronization as well

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: ManifoldCF 0.1, ManifoldCF 0.2
• Fix Version/s: ManifoldCF 1.5
• Component/s: Framework core
• Assignee: Karl Wright
• Reporter: Karl Wright
• Created: 22/Feb/10 16:52
• Updated: 13/Nov/13 18:03
• Resolved: 13/Nov/13 18:03

Description

The current implementation relies on the file system to synchronize activity between various LCF processes. This has several downsides: first, it is possible to get the file system into a state that is corrupted (by killing processes); second, this limits the future ability to spread crawler workload over multiple machines.

It should be reasonably straightforward, and probably more resilient, to introduce a "synchronization process", which all other LCF processes talk to in order to manage locks, shared data, and other synchronization activities.

Issue Links

• is depended upon by: CONNECTORS-781 Fault-Tolerant Setup for ManifoldCF Agent.