[IGNITE-7366] Affinity assignment exception in service processor during multiple nodes join - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 2.3
Fix Version/s: 2.7
Component/s: compute
Labels:
None

Description

When two nodes which are deploying services join at the same time, and exception is observed:

SEVERE: Error when executing service: null
java.lang.IllegalStateException: Getting affinity for topology version earlier than affinity is calculated [locNode=TcpDiscoveryNode [id=245d4bec-0384-4808-b66d-d2340930207f..., discPort=37500, order=2, intOrder=2, lastExchangeTime=1515394551283, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], grp=ignite-sys-cache, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=0], head=AffinityTopologyVersion [topVer=4, minorTopVer=0], history=[AffinityTopologyVersion [topVer=2, minorTopVer=0], AffinityTopologyVersion [topVer=2, minorTopVer=1], AffinityTopologyVersion [topVer=4, minorTopVer=0]]]
at org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.cachedAffinity(GridAffinityAssignmentCache.java:514)
at org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.nodes(GridAffinityAssignmentCache.java:419)
at org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.nodesByPartition(GridCacheAffinityManager.java:220)
at org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primaryByPartition(GridCacheAffinityManager.java:256)
at org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primaryByKey(GridCacheAffinityManager.java:247)
at org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primaryByKey(GridCacheAffinityManager.java:271)
at org.apache.ignite.internal.processors.service.GridServiceProcessor$TopologyListener$1.run0(GridServiceProcessor.java:1771)
at org.apache.ignite.internal.processors.service.GridServiceProcessor$DepRunnable.run(GridServiceProcessor.java:1958)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

This may be caused by exchange merges. There are 4 nodes joining topology. When nodes 3 and 4 join at the same time, exchanges for [3, 0] and [4, 0] are merged. But, TopologyListener in service processor is notified about topVer [3, 0], for which there is no affinity because exchange has already moved forward to [4, 0].