[IGNITE-8390] WAL historical rebalance is not able to process cache.remove() updates

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.4
• Fix Version/s: 2.5
• Component/s: cache
• Assignee: Pavel Kovalenko
• Reporter: Pavel Kovalenko
• Created: 25/Apr/18 16:58
• Updated: 26/Apr/18 13:11
• Resolved: 26/Apr/18 13:10

Description

WAL historical rebalance fails on supplier when process entry remove with following assertion:

java.lang.AssertionError: GridCacheEntryInfo [key=KeyCacheObjectImpl [part=-1, val=2, hasValBytes=true], cacheId=94416770, val=null, ttl=0, expireTime=0, ver=GridCacheVersion [topVer=136155335, order=1524675346187, nodeOrder=1], isNew=false, deleted=false]
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyMessage.addEntry0(GridDhtPartitionSupplyMessage.java:220)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplier.handleDemandMessage(GridDhtPartitionSupplier.java:381)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleDemandMessage(GridDhtPreloader.java:364)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:379)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:364)
	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054)
	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:99)
	at org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1603)
	at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
	at org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
	at org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2752)
	at org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1516)
	at org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:125)
	at org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1485)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Obviously this assertion will work correctly only for full rebalance. We should either soft assertion for historical rebalance case or disable it.
In case of disabled assertion everything works well and rebalance finished properly.

Issue Links

• links to: GitHub Pull Request #3917