Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-1731

[Test Failed] GridCacheTxNodeFailureSelfTest.testPrimaryNodeFailureBackupRollbackOptimistic

    XMLWordPrintableJSON

Details

    • Test
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Test failed on TC. Could not reproduce locally.

      Stack trace:

      java.lang.NullPointerException: null
          at org.apache.ignite.internal.processors.affinity.GridAffinityProcessor$AffinityInfo.access$1800(GridAffinityProcessor.java:537)
          at org.apache.ignite.internal.processors.affinity.GridAffinityProcessor$CacheAffinityProxy.mapKeyToPrimaryAndBackups(GridAffinityProcessor.java:833)
          at org.apache.ignite.testframework.junits.common.GridCommonAbstractTest.backupNode(GridCommonAbstractTest.java:970)
          at org.apache.ignite.internal.processors.cache.distributed.dht.GridCacheTxNodeFailureSelfTest.checkPrimaryNodeFailureBackupCommit(GridCacheTxNodeFailureSelfTest.java:206)
          at org.apache.ignite.internal.processors.cache.distributed.dht.GridCacheTxNodeFailureSelfTest.testPrimaryNodeFailureBackupRollbackOptimistic(GridCacheTxNodeFailureSelfTest.java:140)
      ------- Stdout: -------
      
      

      Full log:

      java.lang.NullPointerException: null
          at org.apache.ignite.internal.processors.affinity.GridAffinityProcessor$AffinityInfo.access$1800(GridAffinityProcessor.java:537)
          at org.apache.ignite.internal.processors.affinity.GridAffinityProcessor$CacheAffinityProxy.mapKeyToPrimaryAndBackups(GridAffinityProcessor.java:833)
          at org.apache.ignite.testframework.junits.common.GridCommonAbstractTest.backupNode(GridCommonAbstractTest.java:970)
          at org.apache.ignite.internal.processors.cache.distributed.dht.GridCacheTxNodeFailureSelfTest.checkPrimaryNodeFailureBackupCommit(GridCacheTxNodeFailureSelfTest.java:206)
          at org.apache.ignite.internal.processors.cache.distributed.dht.GridCacheTxNodeFailureSelfTest.testPrimaryNodeFailureBackupRollbackOptimistic(GridCacheTxNodeFailureSelfTest.java:140)
      ------- Stdout: -------
      [02:10:47,574][INFO ][main][root] >>> Starting test class: GridCacheTxNodeFailureSelfTest <<<
      [02:10:47,654][INFO ][main][root] >>> Starting test: testPrimaryNodeFailureBackupRollbackOptimistic <<<
      [02:10:47,679][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] 
      
      >>>    __________  ________________  
      >>>   /  _/ ___/ |/ /  _/_  __/ __/  
      >>>  _/ // (7 7    // /  / / / _/    
      >>> /___/\___/_/|_/___/ /_/ /___/   
      >>> 
      >>> ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV
      >>> 2015 Copyright(C) Apache Software Foundation
      >>> 
      >>> Ignite documentation: http://ignite.apache.org
      
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] Config URL: n/a
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] Daemon mode: off
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] OS: Linux 3.13.0-32-generic amd64
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] OS user: teamcity
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] Language runtime: Java Platform API Specification ver. 1.7
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] VM information: Java(TM) SE Runtime Environment 1.7.0_80-b15 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 24.80-b11
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] VM total memory: 2.8GB
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] Remote Management [restart: off, REST: off, JMX (remote: off)]
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] IGNITE_HOME=/home/teamcity/BuildAgent/work/871ff4a46e450b13
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] VM arguments: [-DJAVA_HOME=/usr/lib/jvm/java-7-oracle, -Dagent.home.dir=/home/teamcity/BuildAgent, -Dagent.name=teamcity-7, -Dagent.ownPort=9090, -Dagent.work.dir=/home/teamcity/BuildAgent/work, -Dbuild.number=3460, -Dbuild.vcs.number=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dbuild.vcs.number.1=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dbuild.vcs.number.ApacheIgniteMirrorOnGitHub=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dclassworlds.conf=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.m2.conf, -Dcom.jetbrains.maven.watcher.report.file=/home/teamcity/BuildAgent/temp/buildTmp/maven-build-info.xml, -Djava.io.tmpdir=/home/teamcity/BuildAgent/temp/buildTmp, -Dmaven.home=/home/teamcity/BuildAgent/tools/maven3, -Dmaven.repo.local=/home/teamcity/.m2/repository, -Dteamcity.agent.cpuBenchmark=662, -Dteamcity.agent.dotnet.agent_url=http://localhost:9090/RPC2, -Dteamcity.agent.dotnet.build_id=553324, -Dteamcity.auth.password=ruAFOHszAAKHaEYUZeOOGFUGg5ZDcUul, -Dteamcity.auth.userId=TeamCityBuildId=553324, -Dteamcity.build.changedFiles.file=/home/teamcity/BuildAgent/temp/buildTmp/changedFiles8523954552061447373.txt, -Dteamcity.build.checkoutDir=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -Dteamcity.build.id=553324, -Dteamcity.build.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.build14098065591533310.properties, -Dteamcity.build.tempDir=/home/teamcity/BuildAgent/temp/buildTmp, -Dteamcity.build.workingDir=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -Dteamcity.buildConfName=Ignite Cache Failover, -Dteamcity.buildType.id=Ignite_DataGridFailover, -Dteamcity.configuration.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.config2571196353152096152.properties, -Dteamcity.maven.watcher.home=/home/teamcity/BuildAgent/plugins/mavenPlugin/maven-watcher, -Dteamcity.projectName=Ignite Tests, -Dteamcity.runner.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.runner8113397046083155812.properties, -Dteamcity.tests.recentlyFailedTests.file=/home/teamcity/BuildAgent/temp/buildTmp/testsToRunFirst1532190590777282020.txt, -Dteamcity.version=8.1.4 (build 30168), -ea, -XX:MaxPermSize=1024m, -Xms1g, -Xmx3g, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -XX:+AggressiveOpts, -DIGNITE_TEST_HOME=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -DIGNITE_UPDATE_NOTIFIER=false, -DIGNITE_NO_DISCO_ORDER=true, -DIGNITE_QUIET=false]
      [02:10:47,682][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] Configured caches ['ignite-marshaller-sys-cache', 'ignite-sys-cache', 'ignite-atomics-sys-cache', 'default']
      [02:10:47,685][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] 3-rd party licenses can be found at: /home/teamcity/BuildAgent/work/871ff4a46e450b13/libs/licenses
      [02:10:47,687][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] Non-loopback local IPs: 192.168.2.17, fe80:0:0:0:ca60:ff:fe68:5868%2
      [02:10:47,687][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] Enabled local MACs: C86000685868
      [02:10:47,687][INFO ][test-runner][IgnitePluginProcessor] Configured plugins:
      [02:10:47,688][INFO ][test-runner][IgnitePluginProcessor]   ^-- None
      [02:10:47,688][INFO ][test-runner][IgnitePluginProcessor] 
      [02:10:47,738][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] IPC shared memory server endpoint started [port=48100, tokDir=/home/teamcity/BuildAgent/temp/buildTmp/ignite/work/ipc/shmem/005db780-6cb1-420a-8095-9f6a57520000-30523]
      [02:10:47,738][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] Successfully bound shared memory communication to TCP port [port=48100, locHost=/127.0.0.1]
      [02:10:47,739][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] Successfully bound to TCP port [port=47100, locHost=/127.0.0.1]
      [02:10:47,741][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] Security status [authentication=off, tls/ssl=off]
      [02:10:47,772][INFO ][test-runner][TcpDiscoverySpi] Successfully bound to TCP port [port=47500, localHost=/127.0.0.1]
      [02:10:47,782][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-marshaller-sys-cache, mode=REPLICATED]
      [02:10:47,783][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-atomics-sys-cache, mode=PARTITIONED]
      [02:10:47,786][INFO ][test-runner][GridCacheProcessor] Started cache [name=default, mode=PARTITIONED]
      [02:10:47,788][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-sys-cache, mode=REPLICATED]
      [02:10:47,799][INFO ][test-runner][GridDhtPreloader] <ignite-marshaller-sys-cache> Starting rebalancing in SYNC mode: ignite-marshaller-sys-cache
      [02:10:47,799][INFO ][test-runner][GridDhtPreloader] <ignite-atomics-sys-cache> Starting rebalancing in SYNC mode: ignite-atomics-sys-cache
      [02:10:47,799][INFO ][test-runner][GridDhtPreloader] <default> Starting rebalancing in ASYNC mode: null
      [02:10:47,799][INFO ][test-runner][GridDhtPreloader] <ignite-sys-cache> Starting rebalancing in SYNC mode: ignite-sys-cache
      [02:10:47,812][INFO ][preloader-demand-worker-#89239%dht.GridCacheTxNodeFailureSelfTest0%][GridDhtPreloader] <ignite-marshaller-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-marshaller-sys-cache, time=10 ms]
      [02:10:47,812][INFO ][preloader-demand-worker-#89244%dht.GridCacheTxNodeFailureSelfTest0%][GridDhtPreloader] <ignite-atomics-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-atomics-sys-cache, time=10 ms]
      [02:10:47,813][INFO ][preloader-demand-worker-#89248%dht.GridCacheTxNodeFailureSelfTest0%][GridDhtPreloader] <default> Completed rebalancing in ASYNC mode [cache=null, time=10 ms]
      [02:10:47,813][INFO ][preloader-demand-worker-#89252%dht.GridCacheTxNodeFailureSelfTest0%][GridDhtPreloader] <ignite-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-sys-cache, time=10 ms]
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] Performance suggestions for grid 'dht.GridCacheTxNodeFailureSelfTest0' (fix if possible)
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0]   ^-- Disable checkpoints (remove 'checkpointSpi' from configuration)
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0]   ^-- Disable peer class loading (set 'peerClassLoadingEnabled' to false)
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0]   ^-- Disable grid events (remove 'includeEventTypes' from configuration)
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0]   ^-- Decrease number of backups (set 'keyBackups' to 0)
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0]   ^-- Enable ATOMIC mode if not using transactions (set 'atomicityMode' to ATOMIC)
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] 
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] 
      [02:10:47,817][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] 
      
      >>> +------------------------------------------+
      >>> Ignite ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV
      >>> +------------------------------------------+
      >>> OS name: Linux 3.13.0-32-generic amd64
      >>> CPU(s): 4
      >>> Heap: 2.8GB
      >>> VM name: 30523@gg-teamcity-7
      >>> Grid name: dht.GridCacheTxNodeFailureSelfTest0
      >>> Local node [ID=005DB780-6CB1-420A-8095-9F6A57520000, order=1, clientMode=false]
      >>> Local node addresses: [127.0.0.1]
      >>> Local ports: TCP:47100 TCP:47500 TCP:48100 
      
      [02:10:47,817][INFO ][test-runner][GridDiscoveryManager] Topology snapshot [ver=1, servers=1, clients=0, CPUs=4, heap=2.8GB]
      [02:10:47,825][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] 
      
      >>>    __________  ________________  
      >>>   /  _/ ___/ |/ /  _/_  __/ __/  
      >>>  _/ // (7 7    // /  / / / _/    
      >>> /___/\___/_/|_/___/ /_/ /___/   
      >>> 
      >>> ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV
      >>> 2015 Copyright(C) Apache Software Foundation
      >>> 
      >>> Ignite documentation: http://ignite.apache.org
      
      [02:10:47,825][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] Config URL: n/a
      [02:10:47,825][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] Daemon mode: off
      [02:10:47,825][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] OS: Linux 3.13.0-32-generic amd64
      [02:10:47,825][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] OS user: teamcity
      [02:10:47,826][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] Language runtime: Java Platform API Specification ver. 1.7
      [02:10:47,826][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] VM information: Java(TM) SE Runtime Environment 1.7.0_80-b15 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 24.80-b11
      [02:10:47,826][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] VM total memory: 2.8GB
      [02:10:47,826][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] Remote Management [restart: off, REST: off, JMX (remote: off)]
      [02:10:47,826][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] IGNITE_HOME=/home/teamcity/BuildAgent/work/871ff4a46e450b13
      [02:10:47,826][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] VM arguments: [-DJAVA_HOME=/usr/lib/jvm/java-7-oracle, -Dagent.home.dir=/home/teamcity/BuildAgent, -Dagent.name=teamcity-7, -Dagent.ownPort=9090, -Dagent.work.dir=/home/teamcity/BuildAgent/work, -Dbuild.number=3460, -Dbuild.vcs.number=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dbuild.vcs.number.1=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dbuild.vcs.number.ApacheIgniteMirrorOnGitHub=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dclassworlds.conf=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.m2.conf, -Dcom.jetbrains.maven.watcher.report.file=/home/teamcity/BuildAgent/temp/buildTmp/maven-build-info.xml, -Djava.io.tmpdir=/home/teamcity/BuildAgent/temp/buildTmp, -Dmaven.home=/home/teamcity/BuildAgent/tools/maven3, -Dmaven.repo.local=/home/teamcity/.m2/repository, -Dteamcity.agent.cpuBenchmark=662, -Dteamcity.agent.dotnet.agent_url=http://localhost:9090/RPC2, -Dteamcity.agent.dotnet.build_id=553324, -Dteamcity.auth.password=ruAFOHszAAKHaEYUZeOOGFUGg5ZDcUul, -Dteamcity.auth.userId=TeamCityBuildId=553324, -Dteamcity.build.changedFiles.file=/home/teamcity/BuildAgent/temp/buildTmp/changedFiles8523954552061447373.txt, -Dteamcity.build.checkoutDir=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -Dteamcity.build.id=553324, -Dteamcity.build.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.build14098065591533310.properties, -Dteamcity.build.tempDir=/home/teamcity/BuildAgent/temp/buildTmp, -Dteamcity.build.workingDir=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -Dteamcity.buildConfName=Ignite Cache Failover, -Dteamcity.buildType.id=Ignite_DataGridFailover, -Dteamcity.configuration.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.config2571196353152096152.properties, -Dteamcity.maven.watcher.home=/home/teamcity/BuildAgent/plugins/mavenPlugin/maven-watcher, -Dteamcity.projectName=Ignite Tests, -Dteamcity.runner.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.runner8113397046083155812.properties, -Dteamcity.tests.recentlyFailedTests.file=/home/teamcity/BuildAgent/temp/buildTmp/testsToRunFirst1532190590777282020.txt, -Dteamcity.version=8.1.4 (build 30168), -ea, -XX:MaxPermSize=1024m, -Xms1g, -Xmx3g, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -XX:+AggressiveOpts, -DIGNITE_TEST_HOME=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -DIGNITE_UPDATE_NOTIFIER=false, -DIGNITE_NO_DISCO_ORDER=true, -DIGNITE_QUIET=false]
      [02:10:47,826][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] Configured caches ['ignite-marshaller-sys-cache', 'ignite-sys-cache', 'ignite-atomics-sys-cache', 'default']
      [02:10:47,826][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] 3-rd party licenses can be found at: /home/teamcity/BuildAgent/work/871ff4a46e450b13/libs/licenses
      [02:10:47,827][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] Non-loopback local IPs: 192.168.2.17, fe80:0:0:0:ca60:ff:fe68:5868%2
      [02:10:47,827][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] Enabled local MACs: C86000685868
      [02:10:47,827][INFO ][test-runner][IgnitePluginProcessor] Configured plugins:
      [02:10:47,827][INFO ][test-runner][IgnitePluginProcessor]   ^-- None
      [02:10:47,827][INFO ][test-runner][IgnitePluginProcessor] 
      [02:10:47,829][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] IPC shared memory server endpoint started [port=48101, tokDir=/home/teamcity/BuildAgent/temp/buildTmp/ignite/work/ipc/shmem/10107612-e395-4dfb-b705-493472daa001-30523]
      [02:10:47,829][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] Successfully bound shared memory communication to TCP port [port=48101, locHost=/127.0.0.1]
      [02:10:47,830][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] Successfully bound to TCP port [port=47101, locHost=/127.0.0.1]
      [02:10:47,834][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] Security status [authentication=off, tls/ssl=off]
      [02:10:47,840][INFO ][test-runner][TcpDiscoverySpi] Successfully bound to TCP port [port=47502, localHost=/127.0.0.1]
      [02:10:47,887][INFO ][disco-event-worker-#89233%dht.GridCacheTxNodeFailureSelfTest0%][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134247868, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:47,887][INFO ][disco-event-worker-#89233%dht.GridCacheTxNodeFailureSelfTest0%][GridDiscoveryManager] Topology snapshot [ver=2, servers=2, clients=0, CPUs=4, heap=2.8GB]
      [02:10:47,889][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-marshaller-sys-cache, mode=REPLICATED]
      [02:10:47,891][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-atomics-sys-cache, mode=PARTITIONED]
      [02:10:47,895][INFO ][test-runner][GridCacheProcessor] Started cache [name=default, mode=PARTITIONED]
      [02:10:47,897][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-sys-cache, mode=REPLICATED]
      [02:10:48,021][INFO ][test-runner][GridDhtPreloader] <ignite-marshaller-sys-cache> Starting rebalancing in SYNC mode: ignite-marshaller-sys-cache
      [02:10:48,021][INFO ][test-runner][GridDhtPreloader] <ignite-atomics-sys-cache> Starting rebalancing in SYNC mode: ignite-atomics-sys-cache
      [02:10:48,021][INFO ][test-runner][GridDhtPreloader] <default> Starting rebalancing in ASYNC mode: null
      [02:10:48,021][INFO ][test-runner][GridDhtPreloader] <ignite-sys-cache> Starting rebalancing in SYNC mode: ignite-sys-cache
      [02:10:48,025][INFO ][preloader-demand-worker-#89304%dht.GridCacheTxNodeFailureSelfTest1%][GridDhtPreloader] <ignite-marshaller-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-marshaller-sys-cache, time=0 ms]
      [02:10:48,027][INFO ][preloader-demand-worker-#89319%dht.GridCacheTxNodeFailureSelfTest1%][GridDhtPreloader] <ignite-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-sys-cache, time=0 ms]
      [02:10:48,027][INFO ][preloader-demand-worker-#89309%dht.GridCacheTxNodeFailureSelfTest1%][GridDhtPreloader] <ignite-atomics-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-atomics-sys-cache, time=0 ms]
      [02:10:48,103][INFO ][preloader-demand-worker-#89315%dht.GridCacheTxNodeFailureSelfTest1%][GridDhtPreloader] <default> Completed rebalancing in ASYNC mode [cache=null, time=82 ms]
      [02:10:48,106][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] Performance suggestions for grid 'dht.GridCacheTxNodeFailureSelfTest1' (fix if possible)
      [02:10:48,106][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
      [02:10:48,106][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1]   ^-- Disable checkpoints (remove 'checkpointSpi' from configuration)
      [02:10:48,106][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1]   ^-- Disable peer class loading (set 'peerClassLoadingEnabled' to false)
      [02:10:48,106][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1]   ^-- Disable grid events (remove 'includeEventTypes' from configuration)
      [02:10:48,106][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1]   ^-- Decrease number of backups (set 'keyBackups' to 0)
      [02:10:48,106][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1]   ^-- Enable ATOMIC mode if not using transactions (set 'atomicityMode' to ATOMIC)
      [02:10:48,106][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] 
      [02:10:48,106][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
      [02:10:48,106][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] 
      [02:10:48,107][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] 
      
      >>> +------------------------------------------+
      >>> Ignite ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV
      >>> +------------------------------------------+
      >>> OS name: Linux 3.13.0-32-generic amd64
      >>> CPU(s): 4
      >>> Heap: 2.8GB
      >>> VM name: 30523@gg-teamcity-7
      >>> Grid name: dht.GridCacheTxNodeFailureSelfTest1
      >>> Local node [ID=10107612-E395-4DFB-B705-493472DAA001, order=2, clientMode=false]
      >>> Local node addresses: [127.0.0.1]
      >>> Local ports: TCP:47101 TCP:47502 TCP:48101 
      
      [02:10:48,107][INFO ][test-runner][GridDiscoveryManager] Topology snapshot [ver=2, servers=2, clients=0, CPUs=4, heap=2.8GB]
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] 
      
      >>>    __________  ________________  
      >>>   /  _/ ___/ |/ /  _/_  __/ __/  
      >>>  _/ // (7 7    // /  / / / _/    
      >>> /___/\___/_/|_/___/ /_/ /___/   
      >>> 
      >>> ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV
      >>> 2015 Copyright(C) Apache Software Foundation
      >>> 
      >>> Ignite documentation: http://ignite.apache.org
      
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] Config URL: n/a
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] Daemon mode: off
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] OS: Linux 3.13.0-32-generic amd64
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] OS user: teamcity
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] Language runtime: Java Platform API Specification ver. 1.7
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] VM information: Java(TM) SE Runtime Environment 1.7.0_80-b15 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 24.80-b11
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] VM total memory: 2.8GB
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] Remote Management [restart: off, REST: off, JMX (remote: off)]
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] IGNITE_HOME=/home/teamcity/BuildAgent/work/871ff4a46e450b13
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] VM arguments: [-DJAVA_HOME=/usr/lib/jvm/java-7-oracle, -Dagent.home.dir=/home/teamcity/BuildAgent, -Dagent.name=teamcity-7, -Dagent.ownPort=9090, -Dagent.work.dir=/home/teamcity/BuildAgent/work, -Dbuild.number=3460, -Dbuild.vcs.number=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dbuild.vcs.number.1=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dbuild.vcs.number.ApacheIgniteMirrorOnGitHub=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dclassworlds.conf=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.m2.conf, -Dcom.jetbrains.maven.watcher.report.file=/home/teamcity/BuildAgent/temp/buildTmp/maven-build-info.xml, -Djava.io.tmpdir=/home/teamcity/BuildAgent/temp/buildTmp, -Dmaven.home=/home/teamcity/BuildAgent/tools/maven3, -Dmaven.repo.local=/home/teamcity/.m2/repository, -Dteamcity.agent.cpuBenchmark=662, -Dteamcity.agent.dotnet.agent_url=http://localhost:9090/RPC2, -Dteamcity.agent.dotnet.build_id=553324, -Dteamcity.auth.password=ruAFOHszAAKHaEYUZeOOGFUGg5ZDcUul, -Dteamcity.auth.userId=TeamCityBuildId=553324, -Dteamcity.build.changedFiles.file=/home/teamcity/BuildAgent/temp/buildTmp/changedFiles8523954552061447373.txt, -Dteamcity.build.checkoutDir=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -Dteamcity.build.id=553324, -Dteamcity.build.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.build14098065591533310.properties, -Dteamcity.build.tempDir=/home/teamcity/BuildAgent/temp/buildTmp, -Dteamcity.build.workingDir=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -Dteamcity.buildConfName=Ignite Cache Failover, -Dteamcity.buildType.id=Ignite_DataGridFailover, -Dteamcity.configuration.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.config2571196353152096152.properties, -Dteamcity.maven.watcher.home=/home/teamcity/BuildAgent/plugins/mavenPlugin/maven-watcher, -Dteamcity.projectName=Ignite Tests, -Dteamcity.runner.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.runner8113397046083155812.properties, -Dteamcity.tests.recentlyFailedTests.file=/home/teamcity/BuildAgent/temp/buildTmp/testsToRunFirst1532190590777282020.txt, -Dteamcity.version=8.1.4 (build 30168), -ea, -XX:MaxPermSize=1024m, -Xms1g, -Xmx3g, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -XX:+AggressiveOpts, -DIGNITE_TEST_HOME=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -DIGNITE_UPDATE_NOTIFIER=false, -DIGNITE_NO_DISCO_ORDER=true, -DIGNITE_QUIET=false]
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] Configured caches ['ignite-marshaller-sys-cache', 'ignite-sys-cache', 'ignite-atomics-sys-cache', 'default']
      [02:10:48,110][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] 3-rd party licenses can be found at: /home/teamcity/BuildAgent/work/871ff4a46e450b13/libs/licenses
      [02:10:48,112][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] Non-loopback local IPs: 192.168.2.17, fe80:0:0:0:ca60:ff:fe68:5868%2
      [02:10:48,112][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] Enabled local MACs: C86000685868
      [02:10:48,112][INFO ][test-runner][IgnitePluginProcessor] Configured plugins:
      [02:10:48,112][INFO ][test-runner][IgnitePluginProcessor]   ^-- None
      [02:10:48,112][INFO ][test-runner][IgnitePluginProcessor] 
      [02:10:48,113][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] IPC shared memory server endpoint started [port=48102, tokDir=/home/teamcity/BuildAgent/temp/buildTmp/ignite/work/ipc/shmem/209bcf79-b63c-4e44-b97b-38b35c650002-30523]
      [02:10:48,113][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] Successfully bound shared memory communication to TCP port [port=48102, locHost=/127.0.0.1]
      [02:10:48,113][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] Successfully bound to TCP port [port=47102, locHost=/127.0.0.1]
      [02:10:48,143][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] Security status [authentication=off, tls/ssl=off]
      [02:10:48,146][INFO ][test-runner][TcpDiscoverySpi] Successfully bound to TCP port [port=47503, localHost=/127.0.0.1]
      [02:10:48,155][INFO ][disco-event-worker-#89233%dht.GridCacheTxNodeFailureSelfTest0%][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=209bcf79-b63c-4e44-b97b-38b35c650002, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47503], discPort=47503, order=3, intOrder=3, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:48,155][INFO ][disco-event-worker-#89233%dht.GridCacheTxNodeFailureSelfTest0%][GridDiscoveryManager] Topology snapshot [ver=3, servers=3, clients=0, CPUs=4, heap=2.8GB]
      [02:10:48,156][INFO ][disco-event-worker-#89297%dht.GridCacheTxNodeFailureSelfTest1%][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=209bcf79-b63c-4e44-b97b-38b35c650002, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47503], discPort=47503, order=3, intOrder=3, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:48,156][INFO ][disco-event-worker-#89297%dht.GridCacheTxNodeFailureSelfTest1%][GridDiscoveryManager] Topology snapshot [ver=3, servers=3, clients=0, CPUs=4, heap=2.8GB]
      [02:10:48,160][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-marshaller-sys-cache, mode=REPLICATED]
      [02:10:48,162][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-atomics-sys-cache, mode=PARTITIONED]
      [02:10:48,164][INFO ][test-runner][GridCacheProcessor] Started cache [name=default, mode=PARTITIONED]
      [02:10:48,166][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-sys-cache, mode=REPLICATED]
      [02:10:48,217][INFO ][test-runner][GridDhtPreloader] <ignite-marshaller-sys-cache> Starting rebalancing in SYNC mode: ignite-marshaller-sys-cache
      [02:10:48,217][INFO ][test-runner][GridDhtPreloader] <ignite-atomics-sys-cache> Starting rebalancing in SYNC mode: ignite-atomics-sys-cache
      [02:10:48,217][INFO ][test-runner][GridDhtPreloader] <default> Starting rebalancing in ASYNC mode: null
      [02:10:48,217][INFO ][test-runner][GridDhtPreloader] <ignite-sys-cache> Starting rebalancing in SYNC mode: ignite-sys-cache
      [02:10:48,345][INFO ][preloader-demand-worker-#89373%dht.GridCacheTxNodeFailureSelfTest2%][GridDhtPreloader] <ignite-marshaller-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-marshaller-sys-cache, time=131 ms]
      [02:10:48,347][INFO ][preloader-demand-worker-#89387%dht.GridCacheTxNodeFailureSelfTest2%][GridDhtPreloader] <ignite-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-sys-cache, time=131 ms]
      [02:10:48,347][INFO ][preloader-demand-worker-#89377%dht.GridCacheTxNodeFailureSelfTest2%][GridDhtPreloader] <ignite-atomics-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-atomics-sys-cache, time=131 ms]
      [02:10:48,348][INFO ][preloader-demand-worker-#89383%dht.GridCacheTxNodeFailureSelfTest2%][GridDhtPreloader] <default> Completed rebalancing in ASYNC mode [cache=null, time=131 ms]
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] Performance suggestions for grid 'dht.GridCacheTxNodeFailureSelfTest2' (fix if possible)
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2]   ^-- Disable checkpoints (remove 'checkpointSpi' from configuration)
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2]   ^-- Disable peer class loading (set 'peerClassLoadingEnabled' to false)
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2]   ^-- Disable grid events (remove 'includeEventTypes' from configuration)
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2]   ^-- Decrease number of backups (set 'keyBackups' to 0)
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2]   ^-- Enable ATOMIC mode if not using transactions (set 'atomicityMode' to ATOMIC)
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] 
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] 
      [02:10:48,356][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] 
      
      >>> +------------------------------------------+
      >>> Ignite ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV
      >>> +------------------------------------------+
      >>> OS name: Linux 3.13.0-32-generic amd64
      >>> CPU(s): 4
      >>> Heap: 2.8GB
      >>> VM name: 30523@gg-teamcity-7
      >>> Grid name: dht.GridCacheTxNodeFailureSelfTest2
      >>> Local node [ID=209BCF79-B63C-4E44-B97B-38B35C650002, order=3, clientMode=false]
      >>> Local node addresses: [127.0.0.1]
      >>> Local ports: TCP:47102 TCP:47503 TCP:48102 
      
      [02:10:48,356][INFO ][test-runner][GridDiscoveryManager] Topology snapshot [ver=3, servers=3, clients=0, CPUs=4, heap=2.8GB]
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] 
      
      >>>    __________  ________________  
      >>>   /  _/ ___/ |/ /  _/_  __/ __/  
      >>>  _/ // (7 7    // /  / / / _/    
      >>> /___/\___/_/|_/___/ /_/ /___/   
      >>> 
      >>> ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV
      >>> 2015 Copyright(C) Apache Software Foundation
      >>> 
      >>> Ignite documentation: http://ignite.apache.org
      
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] Config URL: n/a
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] Daemon mode: off
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] OS: Linux 3.13.0-32-generic amd64
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] OS user: teamcity
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] Language runtime: Java Platform API Specification ver. 1.7
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] VM information: Java(TM) SE Runtime Environment 1.7.0_80-b15 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 24.80-b11
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] VM total memory: 2.8GB
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] Remote Management [restart: off, REST: off, JMX (remote: off)]
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] IGNITE_HOME=/home/teamcity/BuildAgent/work/871ff4a46e450b13
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] VM arguments: [-DJAVA_HOME=/usr/lib/jvm/java-7-oracle, -Dagent.home.dir=/home/teamcity/BuildAgent, -Dagent.name=teamcity-7, -Dagent.ownPort=9090, -Dagent.work.dir=/home/teamcity/BuildAgent/work, -Dbuild.number=3460, -Dbuild.vcs.number=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dbuild.vcs.number.1=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dbuild.vcs.number.ApacheIgniteMirrorOnGitHub=bc6bf5fcd2c6fd996c3f3d2b1731afed26338091, -Dclassworlds.conf=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.m2.conf, -Dcom.jetbrains.maven.watcher.report.file=/home/teamcity/BuildAgent/temp/buildTmp/maven-build-info.xml, -Djava.io.tmpdir=/home/teamcity/BuildAgent/temp/buildTmp, -Dmaven.home=/home/teamcity/BuildAgent/tools/maven3, -Dmaven.repo.local=/home/teamcity/.m2/repository, -Dteamcity.agent.cpuBenchmark=662, -Dteamcity.agent.dotnet.agent_url=http://localhost:9090/RPC2, -Dteamcity.agent.dotnet.build_id=553324, -Dteamcity.auth.password=ruAFOHszAAKHaEYUZeOOGFUGg5ZDcUul, -Dteamcity.auth.userId=TeamCityBuildId=553324, -Dteamcity.build.changedFiles.file=/home/teamcity/BuildAgent/temp/buildTmp/changedFiles8523954552061447373.txt, -Dteamcity.build.checkoutDir=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -Dteamcity.build.id=553324, -Dteamcity.build.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.build14098065591533310.properties, -Dteamcity.build.tempDir=/home/teamcity/BuildAgent/temp/buildTmp, -Dteamcity.build.workingDir=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -Dteamcity.buildConfName=Ignite Cache Failover, -Dteamcity.buildType.id=Ignite_DataGridFailover, -Dteamcity.configuration.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.config2571196353152096152.properties, -Dteamcity.maven.watcher.home=/home/teamcity/BuildAgent/plugins/mavenPlugin/maven-watcher, -Dteamcity.projectName=Ignite Tests, -Dteamcity.runner.properties.file=/home/teamcity/BuildAgent/temp/buildTmp/teamcity.runner8113397046083155812.properties, -Dteamcity.tests.recentlyFailedTests.file=/home/teamcity/BuildAgent/temp/buildTmp/testsToRunFirst1532190590777282020.txt, -Dteamcity.version=8.1.4 (build 30168), -ea, -XX:MaxPermSize=1024m, -Xms1g, -Xmx3g, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -XX:+AggressiveOpts, -DIGNITE_TEST_HOME=/home/teamcity/BuildAgent/work/871ff4a46e450b13, -DIGNITE_UPDATE_NOTIFIER=false, -DIGNITE_NO_DISCO_ORDER=true, -DIGNITE_QUIET=false]
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] Configured caches ['ignite-marshaller-sys-cache', 'ignite-sys-cache', 'ignite-atomics-sys-cache', 'default']
      [02:10:48,370][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] 3-rd party licenses can be found at: /home/teamcity/BuildAgent/work/871ff4a46e450b13/libs/licenses
      [02:10:48,372][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] Non-loopback local IPs: 192.168.2.17, fe80:0:0:0:ca60:ff:fe68:5868%2
      [02:10:48,372][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] Enabled local MACs: C86000685868
      [02:10:48,372][INFO ][test-runner][IgnitePluginProcessor] Configured plugins:
      [02:10:48,372][INFO ][test-runner][IgnitePluginProcessor]   ^-- None
      [02:10:48,372][INFO ][test-runner][IgnitePluginProcessor] 
      [02:10:48,374][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] IPC shared memory server endpoint started [port=48103, tokDir=/home/teamcity/BuildAgent/temp/buildTmp/ignite/work/ipc/shmem/309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003-30523]
      [02:10:48,374][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] Successfully bound shared memory communication to TCP port [port=48103, locHost=/127.0.0.1]
      [02:10:48,375][INFO ][test-runner][GridCacheTxNodeFailureSelfTest$BanningCommunicationSpi] Successfully bound to TCP port [port=47103, locHost=/127.0.0.1]
      [02:10:48,379][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] Security status [authentication=off, tls/ssl=off]
      [02:10:48,383][INFO ][test-runner][TcpDiscoverySpi] Successfully bound to TCP port [port=47504, localHost=/127.0.0.1]
      [02:10:48,395][INFO ][disco-event-worker-#89233%dht.GridCacheTxNodeFailureSelfTest0%][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248375, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:48,396][INFO ][disco-event-worker-#89233%dht.GridCacheTxNodeFailureSelfTest0%][GridDiscoveryManager] Topology snapshot [ver=4, servers=4, clients=0, CPUs=4, heap=2.8GB]
      [02:10:48,396][INFO ][disco-event-worker-#89297%dht.GridCacheTxNodeFailureSelfTest1%][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:48,396][INFO ][disco-event-worker-#89297%dht.GridCacheTxNodeFailureSelfTest1%][GridDiscoveryManager] Topology snapshot [ver=4, servers=4, clients=0, CPUs=4, heap=2.8GB]
      [02:10:48,397][INFO ][disco-event-worker-#89366%dht.GridCacheTxNodeFailureSelfTest2%][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:48,397][INFO ][disco-event-worker-#89366%dht.GridCacheTxNodeFailureSelfTest2%][GridDiscoveryManager] Topology snapshot [ver=4, servers=4, clients=0, CPUs=4, heap=2.8GB]
      [02:10:48,404][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-marshaller-sys-cache, mode=REPLICATED]
      [02:10:48,406][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-atomics-sys-cache, mode=PARTITIONED]
      [02:10:48,408][INFO ][test-runner][GridCacheProcessor] Started cache [name=default, mode=PARTITIONED]
      [02:10:48,409][INFO ][test-runner][GridCacheProcessor] Started cache [name=ignite-sys-cache, mode=REPLICATED]
      [02:10:48,422][INFO ][test-runner][GridDhtPreloader] <ignite-marshaller-sys-cache> Starting rebalancing in SYNC mode: ignite-marshaller-sys-cache
      [02:10:48,422][INFO ][test-runner][GridDhtPreloader] <ignite-atomics-sys-cache> Starting rebalancing in SYNC mode: ignite-atomics-sys-cache
      [02:10:48,422][INFO ][test-runner][GridDhtPreloader] <default> Starting rebalancing in ASYNC mode: null
      [02:10:48,422][INFO ][test-runner][GridDhtPreloader] <ignite-sys-cache> Starting rebalancing in SYNC mode: ignite-sys-cache
      [02:10:48,505][INFO ][preloader-demand-worker-#89444%dht.GridCacheTxNodeFailureSelfTest3%][GridDhtPreloader] <ignite-marshaller-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-marshaller-sys-cache, time=80 ms]
      [02:10:48,507][INFO ][preloader-demand-worker-#89458%dht.GridCacheTxNodeFailureSelfTest3%][GridDhtPreloader] <ignite-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-sys-cache, time=91 ms]
      [02:10:48,507][INFO ][preloader-demand-worker-#89453%dht.GridCacheTxNodeFailureSelfTest3%][GridDhtPreloader] <default> Completed rebalancing in ASYNC mode [cache=null, time=91 ms]
      [02:10:48,625][INFO ][preloader-demand-worker-#89448%dht.GridCacheTxNodeFailureSelfTest3%][GridDhtPreloader] <ignite-atomics-sys-cache> Completed rebalancing in SYNC mode [cache=ignite-atomics-sys-cache, time=202 ms]
      [02:10:48,635][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] Performance suggestions for grid 'dht.GridCacheTxNodeFailureSelfTest3' (fix if possible)
      [02:10:48,635][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
      [02:10:48,635][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3]   ^-- Disable checkpoints (remove 'checkpointSpi' from configuration)
      [02:10:48,635][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3]   ^-- Disable peer class loading (set 'peerClassLoadingEnabled' to false)
      [02:10:48,635][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3]   ^-- Disable grid events (remove 'includeEventTypes' from configuration)
      [02:10:48,635][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3]   ^-- Decrease number of backups (set 'keyBackups' to 0)
      [02:10:48,635][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3]   ^-- Enable ATOMIC mode if not using transactions (set 'atomicityMode' to ATOMIC)
      [02:10:48,635][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] 
      [02:10:48,635][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
      [02:10:48,635][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] 
      [02:10:48,636][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] 
      
      >>> +------------------------------------------+
      >>> Ignite ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV
      >>> +------------------------------------------+
      >>> OS name: Linux 3.13.0-32-generic amd64
      >>> CPU(s): 4
      >>> Heap: 2.8GB
      >>> VM name: 30523@gg-teamcity-7
      >>> Grid name: dht.GridCacheTxNodeFailureSelfTest3
      >>> Local node [ID=309B4EF4-7CB1-43B5-A3C8-AB33AE8F7003, order=4, clientMode=false]
      >>> Local node addresses: [127.0.0.1]
      >>> Local ports: TCP:47103 TCP:47504 TCP:48103 
      
      [02:10:48,636][INFO ][test-runner][GridDiscoveryManager] Topology snapshot [ver=4, servers=4, clients=0, CPUs=4, heap=2.8GB]
      [02:10:53,057][INFO ][test-runner][root] Grid 0: 005db780-6cb1-420a-8095-9f6a57520000
      [02:10:53,057][INFO ][test-runner][root] Grid 1: 10107612-e395-4dfb-b705-493472daa001
      [02:10:53,057][INFO ][test-runner][root] Grid 2: 209bcf79-b63c-4e44-b97b-38b35c650002
      [02:10:53,057][INFO ][test-runner][root] Grid 3: 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003
      [02:10:53,097][INFO ][test-runner][root] >>> Stopping grid [name=dht.IgniteAtomicLongChangingTopologySelfTest4, id=40403899-e6f3-4e14-ad30-e976820f2004]
      [02:10:53,105][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-marshaller-sys-cache
      [02:10:53,105][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-sys-cache
      [02:10:53,105][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-atomics-sys-cache
      [02:10:53,105][INFO ][test-runner][GridDeploymentLocalStore] Removed undeployed class: GridDeployment [ts=1445134242360, depMode=SHARED, clsLdr=IsolatedClassLoader{roleName='test'}, clsLdrId=b1f98548051-40403899-e6f3-4e14-ad30-e976820f2004, userVer=0, loc=true, sampleClsName=org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap, pendingUndeploy=false, undeployed=true, usage=0]
      [02:10:53,105][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap
      [02:10:53,105][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.GridCacheIoManager$1$1$1
      [02:10:53,105][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.datastructures.CacheDataStructuresConfigurationKey
      [02:10:53,105][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap
      [02:10:53,105][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ResendTimeoutObject$1
      [02:10:53,105][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.GridTopic$T7
      [02:10:53,105][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.task.GridTaskProcessor$TaskDiscoveryListener$1
      [02:10:53,107][INFO ][test-runner][IgniteAtomicLongChangingTopologySelfTest4] 
      
      >>> +-----------------------------------------------------+
      >>> Ignite ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV stopped OK
      >>> +-----------------------------------------------------+
      >>> Grid name: dht.IgniteAtomicLongChangingTopologySelfTest4
      >>> Grid uptime: 00:00:10:726
      
      
      [02:10:53,108][INFO ][test-runner][root] >>> Stopping grid [name=dht.GridCacheTxNodeFailureSelfTest1, id=10107612-e395-4dfb-b705-493472daa001]
      [02:10:53,132][INFO ][disco-event-worker-#89233%dht.GridCacheTxNodeFailureSelfTest0%][GridDiscoveryManager] Node left topology: TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134247868, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:53,132][INFO ][disco-event-worker-#89233%dht.GridCacheTxNodeFailureSelfTest0%][GridDiscoveryManager] Topology snapshot [ver=5, servers=3, clients=0, CPUs=4, heap=2.8GB]
      [02:10:53,133][INFO ][test-runner][GridCacheProcessor] Stopped cache: null
      [02:10:53,133][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-marshaller-sys-cache
      [02:10:53,133][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-sys-cache
      [02:10:53,133][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-atomics-sys-cache
      [02:10:53,133][INFO ][test-runner][GridDeploymentLocalStore] Removed undeployed class: GridDeployment [ts=1445134248020, depMode=SHARED, clsLdr=IsolatedClassLoader{roleName='test'}, clsLdrId=3e1c8548051-10107612-e395-4dfb-b705-493472daa001, userVer=0, loc=true, sampleClsName=org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap, pendingUndeploy=false, undeployed=true, usage=0]
      [02:10:53,133][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap
      [02:10:53,133][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap
      [02:10:53,133][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ResendTimeoutObject$1
      [02:10:53,134][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.GridTopic$T7
      [02:10:53,146][INFO ][disco-event-worker-#89366%dht.GridCacheTxNodeFailureSelfTest2%][GridDiscoveryManager] Node left topology: TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:53,146][INFO ][disco-event-worker-#89366%dht.GridCacheTxNodeFailureSelfTest2%][GridDiscoveryManager] Topology snapshot [ver=5, servers=3, clients=0, CPUs=4, heap=2.8GB]
      [02:10:53,147][INFO ][disco-event-worker-#89437%dht.GridCacheTxNodeFailureSelfTest3%][GridDiscoveryManager] Node left topology: TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:53,147][INFO ][disco-event-worker-#89437%dht.GridCacheTxNodeFailureSelfTest3%][GridDiscoveryManager] Topology snapshot [ver=5, servers=3, clients=0, CPUs=4, heap=2.8GB]
      [02:10:53,165][INFO ][test-runner][GridCacheTxNodeFailureSelfTest1] 
      
      >>> +-----------------------------------------------------+
      >>> Ignite ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV stopped OK
      >>> +-----------------------------------------------------+
      >>> Grid name: dht.GridCacheTxNodeFailureSelfTest1
      >>> Grid uptime: 00:00:05:55
      
      
      [02:10:53,166][INFO ][test-runner][root] >>> Stopping grid [name=dht.GridCacheTxNodeFailureSelfTest0, id=005db780-6cb1-420a-8095-9f6a57520000]
      [02:10:53,171][INFO ][disco-event-worker-#89366%dht.GridCacheTxNodeFailureSelfTest2%][GridDiscoveryManager] Node left topology: TcpDiscoveryNode [id=005db780-6cb1-420a-8095-9f6a57520000, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:53,171][INFO ][disco-event-worker-#89366%dht.GridCacheTxNodeFailureSelfTest2%][GridDiscoveryManager] Topology snapshot [ver=6, servers=2, clients=0, CPUs=4, heap=2.8GB]
      [02:10:53,172][INFO ][disco-event-worker-#89437%dht.GridCacheTxNodeFailureSelfTest3%][GridDiscoveryManager] Node left topology: TcpDiscoveryNode [id=005db780-6cb1-420a-8095-9f6a57520000, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:53,172][INFO ][disco-event-worker-#89437%dht.GridCacheTxNodeFailureSelfTest3%][GridDiscoveryManager] Topology snapshot [ver=6, servers=2, clients=0, CPUs=4, heap=2.8GB]
      [02:10:53,172][INFO ][test-runner][GridCacheProcessor] Stopped cache: null
      [02:10:53,172][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-marshaller-sys-cache
      [02:10:53,172][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-sys-cache
      [02:10:53,173][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-atomics-sys-cache
      [02:10:53,173][INFO ][test-runner][GridDeploymentLocalStore] Removed undeployed class: GridDeployment [ts=1445134248009, depMode=SHARED, clsLdr=IsolatedClassLoader{roleName='test'}, clsLdrId=1e1c8548051-005db780-6cb1-420a-8095-9f6a57520000, userVer=0, loc=true, sampleClsName=org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap, pendingUndeploy=false, undeployed=true, usage=0]
      [02:10:53,173][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap
      [02:10:53,173][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ResendTimeoutObject$1
      [02:10:53,173][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.GridTopic$T7
      [02:10:53,173][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.task.GridTaskProcessor$TaskDiscoveryListener$1
      [02:10:53,181][INFO ][test-runner][GridCacheTxNodeFailureSelfTest0] 
      
      >>> +-----------------------------------------------------+
      >>> Ignite ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV stopped OK
      >>> +-----------------------------------------------------+
      >>> Grid name: dht.GridCacheTxNodeFailureSelfTest0
      >>> Grid uptime: 00:00:05:369
      
      
      [02:10:53,184][INFO ][test-runner][root] >>> Stopping grid [name=dht.GridCacheTxNodeFailureSelfTest3, id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003]
      [02:10:53,235][INFO ][disco-event-worker-#89366%dht.GridCacheTxNodeFailureSelfTest2%][GridDiscoveryManager] Node left topology: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]
      [02:10:53,235][INFO ][disco-event-worker-#89366%dht.GridCacheTxNodeFailureSelfTest2%][GridDiscoveryManager] Topology snapshot [ver=7, servers=1, clients=0, CPUs=4, heap=2.8GB]
      [02:10:53,240][INFO ][test-runner][GridCacheProcessor] Stopped cache: null
      [02:10:53,240][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-marshaller-sys-cache
      [02:10:53,240][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-sys-cache
      [02:10:53,240][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-atomics-sys-cache
      [02:10:53,240][INFO ][test-runner][GridDeploymentLocalStore] Removed undeployed class: GridDeployment [ts=1445134248415, depMode=SHARED, clsLdr=IsolatedClassLoader{roleName='test'}, clsLdrId=c9bc8548051-309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, userVer=0, loc=true, sampleClsName=org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap, pendingUndeploy=false, undeployed=true, usage=0]
      [02:10:53,240][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap
      [02:10:53,240][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap
      [02:10:53,240][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ResendTimeoutObject$1
      [02:10:53,240][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.GridTopic$T7
      [02:10:53,240][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.task.GridTaskProcessor$TaskDiscoveryListener$1
      [02:10:53,244][INFO ][test-runner][GridCacheTxNodeFailureSelfTest3] 
      
      >>> +-----------------------------------------------------+
      >>> Ignite ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV stopped OK
      >>> +-----------------------------------------------------+
      >>> Grid name: dht.GridCacheTxNodeFailureSelfTest3
      >>> Grid uptime: 00:00:04:610
      
      
      [02:10:53,245][INFO ][test-runner][root] >>> Stopping grid [name=dht.GridCacheTxNodeFailureSelfTest2, id=209bcf79-b63c-4e44-b97b-38b35c650002]
      [02:10:53,258][INFO ][test-runner][GridCacheProcessor] Stopped cache: null
      [02:10:53,258][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-marshaller-sys-cache
      [02:10:53,258][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-sys-cache
      [02:10:53,258][INFO ][test-runner][GridCacheProcessor] Stopped cache: ignite-atomics-sys-cache
      [02:10:53,258][INFO ][test-runner][GridDeploymentLocalStore] Removed undeployed class: GridDeployment [ts=1445134248213, depMode=SHARED, clsLdr=IsolatedClassLoader{roleName='test'}, clsLdrId=296c8548051-209bcf79-b63c-4e44-b97b-38b35c650002, userVer=0, loc=true, sampleClsName=org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap, pendingUndeploy=false, undeployed=true, usage=0]
      [02:10:53,258][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionFullMap
      [02:10:53,258][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap
      [02:10:53,258][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ResendTimeoutObject$1
      [02:10:53,258][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.GridTopic$T7
      [02:10:53,258][INFO ][test-runner][GridDeploymentLocalStore] Class locally undeployed: class org.apache.ignite.internal.processors.task.GridTaskProcessor$TaskDiscoveryListener$1
      [02:10:53,264][INFO ][test-runner][GridCacheTxNodeFailureSelfTest2] 
      
      >>> +-----------------------------------------------------+
      >>> Ignite ver. 1.5.0-SNAPSHOT#19700101-sha1:DEV stopped OK
      >>> +-----------------------------------------------------+
      >>> Grid name: dht.GridCacheTxNodeFailureSelfTest2
      >>> Grid uptime: 00:00:04:903
      
      
      [02:10:53,271][INFO ][main][root] >>> Stopping test: testPrimaryNodeFailureBackupRollbackOptimistic in 5617 ms <<<
      ------- Stderr: -------
      [02:10:47,682][WARN ][test-runner][GridCacheTxNodeFailureSelfTest0] Peer class loading is enabled (disable it in production for performance and deployment consistency reasons)
      [02:10:47,741][WARN ][test-runner][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival).
      [02:10:47,741][WARN ][test-runner][NoopSwapSpaceSpi] Swap space is disabled. To enable use FileSwapSpaceSpi.
      [02:10:47,826][WARN ][test-runner][GridCacheTxNodeFailureSelfTest1] Peer class loading is enabled (disable it in production for performance and deployment consistency reasons)
      [02:10:47,834][WARN ][test-runner][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival).
      [02:10:47,834][WARN ][test-runner][NoopSwapSpaceSpi] Swap space is disabled. To enable use FileSwapSpaceSpi.
      [02:10:48,110][WARN ][test-runner][GridCacheTxNodeFailureSelfTest2] Peer class loading is enabled (disable it in production for performance and deployment consistency reasons)
      [02:10:48,142][WARN ][test-runner][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival).
      [02:10:48,142][WARN ][test-runner][NoopSwapSpaceSpi] Swap space is disabled. To enable use FileSwapSpaceSpi.
      [02:10:48,158][WARN ][test-runner][GridCacheTxNodeFailureSelfTest2] Attempting to start more nodes than physical RAM available on current host (this can cause significant slowdown)
      [02:10:48,370][WARN ][test-runner][GridCacheTxNodeFailureSelfTest3] Peer class loading is enabled (disable it in production for performance and deployment consistency reasons)
      [02:10:48,379][WARN ][test-runner][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival).
      [02:10:48,379][WARN ][test-runner][NoopSwapSpaceSpi] Swap space is disabled. To enable use FileSwapSpaceSpi.
      [02:10:48,403][WARN ][test-runner][GridCacheTxNodeFailureSelfTest3] Attempting to start more nodes than physical RAM available on current host (this can cause significant slowdown)
      [02:10:48,636][WARN ][test-runner][root] Waiting for topology map update [grid=dht.GridCacheTxNodeFailureSelfTest1, cache=ignite-atomics-sys-cache, cacheId=689859866, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], topFut=GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], topVer=4, nodeId8=10107612, msg=Node joined: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], type=NODE_JOINED, tstamp=1445134248395], rcvdIds=GridConcurrentHashSet [elements=[]], rmtIds=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], nodeId=309b4ef4, evt=NODE_JOINED], init=true, ready=true, replied=false, added=true, initFut=GridFutureAdapter [resFlag=2, res=true, startTime=1445134248395, endTime=1445134248405, ignoreInterrupts=false, lsnr=null, state=DONE], topSnapshot=null, lastVer=null, partReleaseFut=GridCompoundFuture [lsnrCalls=3, finished=true, rdc=null, init=true, res=java.util.concurrent.atomic.AtomicMarkableReference@6da76db3, err=null, done=true, cancelled=false, err=null, futs=[true, true, true]], skipPreload=false, clientOnlyExchange=false, initTs=1445134248395, oldest=005db780-6cb1-420a-8095-9f6a57520000, oldestOrder=1, evtLatch=0, remaining=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], super=GridFutureAdapter [resFlag=2, res=AffinityTopologyVersion [topVer=4, minorTopVer=0], startTime=1445134248395, endTime=1445134248415, ignoreInterrupts=false, lsnr=null, state=DONE]], p=2, affNodesCnt=1, ownersCnt=2, affNodes=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], owners=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], TcpDiscoveryNode [id=209bcf79-b63c-4e44-b97b-38b35c650002, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47503], discPort=47503, order=3, intOrder=3, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], locNode=TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134248627, loc=true, ver=1.5.0#19700101-sha1:00000000, isClient=false]]
      [02:10:48,837][WARN ][test-runner][root] Waiting for topology map update [grid=dht.GridCacheTxNodeFailureSelfTest1, cache=ignite-atomics-sys-cache, cacheId=689859866, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], topFut=GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], topVer=4, nodeId8=10107612, msg=Node joined: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], type=NODE_JOINED, tstamp=1445134248395], rcvdIds=GridConcurrentHashSet [elements=[]], rmtIds=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], nodeId=309b4ef4, evt=NODE_JOINED], init=true, ready=true, replied=false, added=true, initFut=GridFutureAdapter [resFlag=2, res=true, startTime=1445134248395, endTime=1445134248405, ignoreInterrupts=false, lsnr=null, state=DONE], topSnapshot=null, lastVer=null, partReleaseFut=GridCompoundFuture [lsnrCalls=3, finished=true, rdc=null, init=true, res=java.util.concurrent.atomic.AtomicMarkableReference@6da76db3, err=null, done=true, cancelled=false, err=null, futs=[true, true, true]], skipPreload=false, clientOnlyExchange=false, initTs=1445134248395, oldest=005db780-6cb1-420a-8095-9f6a57520000, oldestOrder=1, evtLatch=0, remaining=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], super=GridFutureAdapter [resFlag=2, res=AffinityTopologyVersion [topVer=4, minorTopVer=0], startTime=1445134248395, endTime=1445134248415, ignoreInterrupts=false, lsnr=null, state=DONE]], p=2, affNodesCnt=1, ownersCnt=2, affNodes=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], owners=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], TcpDiscoveryNode [id=209bcf79-b63c-4e44-b97b-38b35c650002, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47503], discPort=47503, order=3, intOrder=3, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], locNode=TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134248778, loc=true, ver=1.5.0#19700101-sha1:00000000, isClient=false]]
      [02:10:49,840][WARN ][test-runner][root] Waiting for topology map update [grid=dht.GridCacheTxNodeFailureSelfTest1, cache=ignite-atomics-sys-cache, cacheId=689859866, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], topFut=GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], topVer=4, nodeId8=10107612, msg=Node joined: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], type=NODE_JOINED, tstamp=1445134248395], rcvdIds=GridConcurrentHashSet [elements=[]], rmtIds=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], nodeId=309b4ef4, evt=NODE_JOINED], init=true, ready=true, replied=false, added=true, initFut=GridFutureAdapter [resFlag=2, res=true, startTime=1445134248395, endTime=1445134248405, ignoreInterrupts=false, lsnr=null, state=DONE], topSnapshot=null, lastVer=null, partReleaseFut=GridCompoundFuture [lsnrCalls=3, finished=true, rdc=null, init=true, res=java.util.concurrent.atomic.AtomicMarkableReference@6da76db3, err=null, done=true, cancelled=false, err=null, futs=[true, true, true]], skipPreload=false, clientOnlyExchange=false, initTs=1445134248395, oldest=005db780-6cb1-420a-8095-9f6a57520000, oldestOrder=1, evtLatch=0, remaining=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], super=GridFutureAdapter [resFlag=2, res=AffinityTopologyVersion [topVer=4, minorTopVer=0], startTime=1445134248395, endTime=1445134248415, ignoreInterrupts=false, lsnr=null, state=DONE]], p=2, affNodesCnt=1, ownersCnt=2, affNodes=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], owners=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], TcpDiscoveryNode [id=209bcf79-b63c-4e44-b97b-38b35c650002, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47503], discPort=47503, order=3, intOrder=3, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], locNode=TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134249787, loc=true, ver=1.5.0#19700101-sha1:00000000, isClient=false]]
      [02:10:50,842][WARN ][test-runner][root] Waiting for topology map update [grid=dht.GridCacheTxNodeFailureSelfTest1, cache=ignite-atomics-sys-cache, cacheId=689859866, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], topFut=GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], topVer=4, nodeId8=10107612, msg=Node joined: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], type=NODE_JOINED, tstamp=1445134248395], rcvdIds=GridConcurrentHashSet [elements=[]], rmtIds=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], nodeId=309b4ef4, evt=NODE_JOINED], init=true, ready=true, replied=false, added=true, initFut=GridFutureAdapter [resFlag=2, res=true, startTime=1445134248395, endTime=1445134248405, ignoreInterrupts=false, lsnr=null, state=DONE], topSnapshot=null, lastVer=null, partReleaseFut=GridCompoundFuture [lsnrCalls=3, finished=true, rdc=null, init=true, res=java.util.concurrent.atomic.AtomicMarkableReference@6da76db3, err=null, done=true, cancelled=false, err=null, futs=[true, true, true]], skipPreload=false, clientOnlyExchange=false, initTs=1445134248395, oldest=005db780-6cb1-420a-8095-9f6a57520000, oldestOrder=1, evtLatch=0, remaining=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], super=GridFutureAdapter [resFlag=2, res=AffinityTopologyVersion [topVer=4, minorTopVer=0], startTime=1445134248395, endTime=1445134248415, ignoreInterrupts=false, lsnr=null, state=DONE]], p=2, affNodesCnt=1, ownersCnt=2, affNodes=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], owners=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], TcpDiscoveryNode [id=209bcf79-b63c-4e44-b97b-38b35c650002, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47503], discPort=47503, order=3, intOrder=3, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], locNode=TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134250796, loc=true, ver=1.5.0#19700101-sha1:00000000, isClient=false]]
      [02:10:51,443][WARN ][test-runner][root] Waiting for topology map update [grid=dht.GridCacheTxNodeFailureSelfTest1, cache=ignite-atomics-sys-cache, cacheId=689859866, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], topFut=GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], topVer=4, nodeId8=10107612, msg=Node joined: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], type=NODE_JOINED, tstamp=1445134248395], rcvdIds=GridConcurrentHashSet [elements=[]], rmtIds=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], nodeId=309b4ef4, evt=NODE_JOINED], init=true, ready=true, replied=false, added=true, initFut=GridFutureAdapter [resFlag=2, res=true, startTime=1445134248395, endTime=1445134248405, ignoreInterrupts=false, lsnr=null, state=DONE], topSnapshot=null, lastVer=null, partReleaseFut=GridCompoundFuture [lsnrCalls=3, finished=true, rdc=null, init=true, res=java.util.concurrent.atomic.AtomicMarkableReference@6da76db3, err=null, done=true, cancelled=false, err=null, futs=[true, true, true]], skipPreload=false, clientOnlyExchange=false, initTs=1445134248395, oldest=005db780-6cb1-420a-8095-9f6a57520000, oldestOrder=1, evtLatch=0, remaining=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], super=GridFutureAdapter [resFlag=2, res=AffinityTopologyVersion [topVer=4, minorTopVer=0], startTime=1445134248395, endTime=1445134248415, ignoreInterrupts=false, lsnr=null, state=DONE]], p=2, affNodesCnt=1, ownersCnt=2, affNodes=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], owners=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], TcpDiscoveryNode [id=209bcf79-b63c-4e44-b97b-38b35c650002, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47503], discPort=47503, order=3, intOrder=3, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], locNode=TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134251250, loc=true, ver=1.5.0#19700101-sha1:00000000, isClient=false]]
      [02:10:51,845][WARN ][test-runner][root] Waiting for topology map update [grid=dht.GridCacheTxNodeFailureSelfTest1, cache=ignite-atomics-sys-cache, cacheId=689859866, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], topFut=GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], topVer=4, nodeId8=10107612, msg=Node joined: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], type=NODE_JOINED, tstamp=1445134248395], rcvdIds=GridConcurrentHashSet [elements=[]], rmtIds=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], nodeId=309b4ef4, evt=NODE_JOINED], init=true, ready=true, replied=false, added=true, initFut=GridFutureAdapter [resFlag=2, res=true, startTime=1445134248395, endTime=1445134248405, ignoreInterrupts=false, lsnr=null, state=DONE], topSnapshot=null, lastVer=null, partReleaseFut=GridCompoundFuture [lsnrCalls=3, finished=true, rdc=null, init=true, res=java.util.concurrent.atomic.AtomicMarkableReference@6da76db3, err=null, done=true, cancelled=false, err=null, futs=[true, true, true]], skipPreload=false, clientOnlyExchange=false, initTs=1445134248395, oldest=005db780-6cb1-420a-8095-9f6a57520000, oldestOrder=1, evtLatch=0, remaining=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], super=GridFutureAdapter [resFlag=2, res=AffinityTopologyVersion [topVer=4, minorTopVer=0], startTime=1445134248395, endTime=1445134248415, ignoreInterrupts=false, lsnr=null, state=DONE]], p=2, affNodesCnt=1, ownersCnt=2, affNodes=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], owners=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], TcpDiscoveryNode [id=209bcf79-b63c-4e44-b97b-38b35c650002, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47503], discPort=47503, order=3, intOrder=3, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], locNode=TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134251805, loc=true, ver=1.5.0#19700101-sha1:00000000, isClient=false]]
      [02:10:52,847][WARN ][test-runner][root] Waiting for topology map update [grid=dht.GridCacheTxNodeFailureSelfTest1, cache=ignite-atomics-sys-cache, cacheId=689859866, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], topFut=GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], topVer=4, nodeId8=10107612, msg=Node joined: TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], type=NODE_JOINED, tstamp=1445134248395], rcvdIds=GridConcurrentHashSet [elements=[]], rmtIds=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], nodeId=309b4ef4, evt=NODE_JOINED], init=true, ready=true, replied=false, added=true, initFut=GridFutureAdapter [resFlag=2, res=true, startTime=1445134248395, endTime=1445134248405, ignoreInterrupts=false, lsnr=null, state=DONE], topSnapshot=null, lastVer=null, partReleaseFut=GridCompoundFuture [lsnrCalls=3, finished=true, rdc=null, init=true, res=java.util.concurrent.atomic.AtomicMarkableReference@6da76db3, err=null, done=true, cancelled=false, err=null, futs=[true, true, true]], skipPreload=false, clientOnlyExchange=false, initTs=1445134248395, oldest=005db780-6cb1-420a-8095-9f6a57520000, oldestOrder=1, evtLatch=0, remaining=[209bcf79-b63c-4e44-b97b-38b35c650002, 309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, 005db780-6cb1-420a-8095-9f6a57520000], super=GridFutureAdapter [resFlag=2, res=AffinityTopologyVersion [topVer=4, minorTopVer=0], startTime=1445134248395, endTime=1445134248415, ignoreInterrupts=false, lsnr=null, state=DONE]], p=2, affNodesCnt=1, ownersCnt=2, affNodes=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], owners=[TcpDiscoveryNode [id=309b4ef4-7cb1-43b5-a3c8-ab33ae8f7003, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47504], discPort=47504, order=4, intOrder=4, lastExchangeTime=1445134248385, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false], TcpDiscoveryNode [id=209bcf79-b63c-4e44-b97b-38b35c650002, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47503], discPort=47503, order=3, intOrder=3, lastExchangeTime=1445134248143, loc=false, ver=1.5.0#19700101-sha1:00000000, isClient=false]], locNode=TcpDiscoveryNode [id=10107612-e395-4dfb-b705-493472daa001, addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47502], discPort=47502, order=2, intOrder=2, lastExchangeTime=1445134252813, loc=true, ver=1.5.0#19700101-sha1:00000000, isClient=false]]
      [02:10:53,047][WARN ][test-runner][root] Finished waiting for topology map update [grid=dht.GridCacheTxNodeFailureSelfTest1, p=2, duration=4411ms]
      [02:10:53,266][ERROR][main][root] Test failed.
      java.lang.NullPointerException
          at org.apache.ignite.internal.processors.affinity.GridAffinityProcessor$AffinityInfo.access$1800(GridAffinityProcessor.java:537)
          at org.apache.ignite.internal.processors.affinity.GridAffinityProcessor$CacheAffinityProxy.mapKeyToPrimaryAndBackups(GridAffinityProcessor.java:833)
          at org.apache.ignite.testframework.junits.common.GridCommonAbstractTest.backupNode(GridCommonAbstractTest.java:970)
          at org.apache.ignite.internal.processors.cache.distributed.dht.GridCacheTxNodeFailureSelfTest.checkPrimaryNodeFailureBackupCommit(GridCacheTxNodeFailureSelfTest.java:206)
          at org.apache.ignite.internal.processors.cache.distributed.dht.GridCacheTxNodeFailureSelfTest.testPrimaryNodeFailureBackupRollbackOptimistic(GridCacheTxNodeFailureSelfTest.java:140)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:606)
          at junit.framework.TestCase.runTest(TestCase.java:176)
          at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:1658)
          at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:112)
          at org.apache.ignite.testframework.junits.GridAbstractTest$4.run(GridAbstractTest.java:1596)
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            ashutak Artem Shutak
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: