[IGNITE-12325] GridCacheMapEntry reservation mechanism is broken with enabled cache store

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8
• Fix Version/s: 2.8
• Component/s: cache
• Assignee: Pavel Kovalenko
• Reporter: Pavel Kovalenko
• Created: 23/Oct/19 12:44
• Updated: 24/Oct/19 13:48
• Resolved: 24/Oct/19 13:47

Description

Entry deferred deletion was disabled after https://issues.apache.org/jira/browse/IGNITE-11704 in transactional caches. However, if cache store is enabled there is a race with cache entry reservation after transactional remove and clear reservation after cache load:

java.lang.AssertionError: GridDhtCacheEntry [rdrs=ReaderId[] [ReaderId [nodeId=96c87c98-2524-4f9e-8a2f-6cfceda00005, msgId=22663371, txFut=null], ReaderId [nodeId=68130805-0dc8-4ef4-abf7-7e7cde800006, msgId=22663375, txFut=null], ReaderId [nodeId=b4a8abce-8d0e-4459-b93a-a734ad600004, msgId=22663370, txFut=null]], part=8, super=GridDistributedCacheEntry [super=GridCacheMapEntry [key=KeyCacheObjectImpl [part=8, val=8, hasValBytes=true], val=null, ver=GridCacheVersion [topVer=0, order=0, nodeOrder=0], hash=8, extras=null, flags=2]]]
	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.clearReserveForLoad(GridCacheMapEntry.java:3616)
	at org.apache.ignite.internal.processors.cache.GridCacheAdapter.clearReservationsIfNeeded(GridCacheAdapter.java:2429)
	at org.apache.ignite.internal.processors.cache.GridCacheAdapter.access$400(GridCacheAdapter.java:179)
	at org.apache.ignite.internal.processors.cache.GridCacheAdapter$18.call(GridCacheAdapter.java:2309)
	at org.apache.ignite.internal.processors.cache.GridCacheAdapter$18.call(GridCacheAdapter.java:2217)
	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6963)
	at org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:844)

The issue can be resolved with enabled deferred delete if cache store is enabled.

Issue Links

• is caused by: IGNITE-11704 Write tombstones during rebalance to get rid of deferred delete buffer
• links to: GitHub Pull Request #7005