[IGNITE-10352] Cache get request can be mapped to node while partition in MOVING state

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8
• Assignee: Dmitriy Govorukhin
• Reporter: Andrey N. Gura
• Created: 20/Nov/18 17:45
• Updated: 03/Jul/20 06:23
• Resolved: 30/Nov/18 13:12

Description

Cache get request can be mapped to node while partition in MOVING state.

It leads to assertions like quoted below and eventually to failure handler action.

java.lang.AssertionError: Wrong ready topology version for invalid partitions response [topVer=AffinityTopologyVersion [topVer=19, minorTopVer=0], req=GridNearSingleGetRequest [futId=1542371256329, key=KeyCacheObjectImpl [part=4595, val=com.sbt.cdm.api.model.published.ae.PublishedOperationCondition, hasValBytes=true], flags=1, topVer=AffinityTopologyVersion [topVer=19, minorTopVer=0], subjId=ea523c3a-4d22-4a3c-b1b0-3137da762021, taskNameHash=0, createTtl=-1, accessTtl=-1, mvccSnapshot=null]]
        at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$6.apply(GridDhtCacheAdapter.java:975)
        at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$6.apply(GridDhtCacheAdapter.java:938)
        at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
        at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:355)
        at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processNearSingleGetRequest(GridDhtCacheAdapter.java:938)
        at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$2.apply(GridDhtTransactionalCacheAdapter.java:152)
        at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$2.apply(GridDhtTransactionalCacheAdapter.java:150)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
        at org.apache.ignite.internal.managers.communication.GridIoManager$ArrayListener.onMessage(GridIoManager.java:2349)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
        at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
        at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505)
        at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:745)

and

java.lang.AssertionError: Got invalid partitions for local node but topology version did not change [topVer=AffinityTopologyVersion [topVer=18, minorTopVer=0], updTopVer=AffinityTopologyVersion [topVer=18, minorTopVer=0], invalidParts=[3032]]
        at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.map(GridPartitionedSingleGetFuture.java:254)
        at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.access$600(GridPartitionedSingleGetFuture.java:69)
        at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture$4.run(GridPartitionedSingleGetFuture.java:774)
        at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6887)
        at org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827)
        at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Issue Links

• is duplicated by: IGNITE-10252 Cache.get() may be mapped to the node with partition state is "MOVING"
• is related to: IGNITE-10274 MVCC: Fix/optimize get from local backup.
• links to: GitHub Pull Request #5451
• links to: GitHub Pull Request #5457
• links to: GitHub Pull Request #5503
• links to: GitHub Pull Request #5505
• links to: GitHub Pull Request #5519