[IGNITE-5692] IgnitePdsCacheRebalancingAbstractTest suite hangs on TC

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.1
• Fix Version/s: 2.1
• Component/s: cache, persistence
• Labels: MakeTeamcityGreenAgain test-fail
• Assignee: Pavel Kovalenko
• Reporter: Pavel Kovalenko
• Created: 04/Jul/17 15:13
• Updated: 27/Jul/17 20:04
• Resolved: 05/Jul/17 18:36

Description

Exact tests are testTopologyChangesWithConstantLoad and testTopologyChangesWithConstantLoadExplicitTx in IgnitePdsTxCacheRebalancingTest

Cause of hanging is unfinished partition map exchange:

[2017-07-04 18:08:31,112][WARN ][test-runner-#1%persistence.IgnitePdsTxCacheRebalancingTest%][root] Waiting for topology map update [igniteInstanceName=persistence.IgnitePdsTxCacheRebalancingTest3, cache=indexed, cacheId=1943292145, topVer=AffinityTopologyVersion [topVer=13, minorTopVer=0], p=0, affNodesCnt=2, ownersCnt=1, affNodes=[c9c01bd9-e15d-4b62-86be-f1a183800003, de98b611-fc6d-4149-857c-9a3253100004], owners=[c9c01bd9-e15d-4b62-86be-f1a183800003], topFut=GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=b71b30b7-2df9-4f35-aeaf-cefd0e900005, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47505], discPort=47505, order=12, intOrder=9, lastExchangeTime=1499180827453, loc=false, ver=2.1.0#19700101-sha1:00000000, isClient=false], topVer=13, nodeId8=c9c01bd9, msg=Node left: TcpDiscoveryNode [id=b71b30b7-2df9-4f35-aeaf-cefd0e900005, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47505], discPort=47505, order=12, intOrder=9, lastExchangeTime=1499180827453, loc=false, ver=2.1.0#19700101-sha1:00000000, isClient=false], type=NODE_LEFT, tstamp=1499180845215], crd=TcpDiscoveryNode [id=1a513139-cfce-48c9-aab0-71b7ecb00000, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1499180791749, loc=false, ver=2.1.0#19700101-sha1:00000000, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=13, minorTopVer=0], nodeId=b71b30b7, evt=NODE_LEFT], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1444700749], init=true, lastVer=null, partReleaseFut=GridCompoundFuture [rdc=null, initFlag=1, lsnrCalls=4, done=true, cancelled=false, err=null, futs=[true, true, true, true]], exchActions=null, affChangeMsg=null, skipPreload=false, clientOnlyExchange=false, initTs=1499180845227, centralizedAff=true, changeGlobalStateE=null, exchangeOnChangeGlobalState=false, forcedRebFut=null, done=true, evtLatch=0, remaining=[89a60809-876d-4ea7-845d-a41ddea00002, de98b611-fc6d-4149-857c-9a3253100004, 1a513139-cfce-48c9-aab0-71b7ecb00000, b2e733e4-35e2-4860-ae90-d0a8b0a00001], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=13, minorTopVer=0], hash=1954469014]], locNode=TcpDiscoveryNode [id=c9c01bd9-e15d-4b62-86be-f1a183800003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47503], discPort=47503, order=10, intOrder=7, lastExchangeTime=1499180911088, loc=true, ver=2.1.0#19700101-sha1:00000000, isClient=false]]