[CAMEL-8010] Race condition in AggregatorProcessor recovery sometimes causes duplicates (still)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.14.0
• Fix Version/s: 2.20.0
• Component/s: camel-core
• Estimated Complexity: Unknown
• Assignee: Andrea Cosentino
• Reporter: Marc Carter
• Created: 07/Nov/14 12:52
• Updated: 24/Aug/17 22:01
• Resolved: 24/Aug/17 09:38

Description

CAMEL-6097 Patched a pretty clear race condition between the completion thread (CT) and recovery thread (RT) but leaves several holes when exercised with a Jdbc repository and a separate aggregation thread (AT).

#1 is relevant to all repository backends.
#2 only affects fully transactional backends

I'm currently taking a look into this bug as its a show-stopper that persistent repositories actually decreases reliability. (Untested) workaround is to add an in-memory idemptotentconsumer immediately after the aggregation.

Here AT starts and completes an aggregation between defensive copy and when RT repo scanning starts. CT then confirms it (in memory ) before repo scanning ends.

AT	RT	CT
	inProg COPY to inProgCopy
repo START x
		repo REMOVE x
		<commit>
		inProg ADD x
	repo SCAN (sees x)
		process x
		repo CONFIRM x
		inProg REMOVE x
		<commit>
	x not inProg or inProgCopy
	process x
	repo CONFIRM x		Fails silently as this is doInTransactionWithoutResult
	<commit>

SQLWarning ignored: SQL state '02000', error code '10000', message [No row was found for FETCH, UPDATE or DELETE; or the result of a query is an empty table.]

Side note: inProgressExchanges is updated by a Synchronisation inside the UOW so is immediately visible although any DB change may not be visible for ages (in threading terms) as the entire transaction must commit first.

Issue Links

• is related to: CAMEL-6097 Race condition in AggregatorProcessor recovery sometimes causes duplicates
• is related to: CAMEL-4271 jdbc aggregation repository - recovery taks and cluster issue
• links to: GitHub Pull Request #1894