[UIMA-3657] DUCC Orchestrator (OR) improved synchronization tracking

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0-Ducc
• Fix Version/s: 1.1.0-Ducc
• Component/s: DUCC
• Assignee: Lou DeGenaro
• Reporter: Lou DeGenaro
• Created: 27/Feb/14 20:35
• Updated: 30/Apr/14 16:28
• Resolved: 30/Apr/14 16:28

Description

The orchestrator currently records to its log some limited and incomplete information about synchronization. This improvement:

1. Instruments all WorkMap synchronizations in the OR
2. Accounts for time blocked and time held
3. Records all new requests for synchronization when current holder exceeds 10 seconds
4. Records all pending requests when current holder releases having held synchronization for > 10 seconds

This is to address the situation, for example, where OR is running albeit slowly. Newly added log messages will hopefully shed light on where the bottlenecks may be.

One theory is that a normally fast resource, such as the filesystem, becomes very slow and bogs down OR while its trying to write its checkpoint dataset. In this case, we'd expect to see the synchronization lock held for a long time by the OR's checkpoint module.