Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-10010

Node halted if second node was stopped, then cache destroyed, then second node returned

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 2.7
    • Fix Version/s: 2.8
    • Component/s: None
    • Labels:
      None

      Description

      1. Start 2 nodes with PDS
      2. Activate cluster
      3. Connect sqlline.
      4. Create table create table t1(a int, b varchar, primary key(a)) with "ATOMICITY=TRANSACTIONAL_SNAPSHOT,backups=1";
      5. Stop node 1
      6. Drop table drop table t1;
      7. Start node 1
      8. Node 2 stopped by handler:

      c:\Work\apache-ignite-2.7.0-SNAPSHOT-bin>bin\ignite.bat server.xml -v -J-DID=1
      Ignite Command Line Startup, ver. 2.7.0-SNAPSHOT#19700101-sha1:DEV
      2018 Copyright(C) Apache Software Foundation
      
      [18:04:22,745][INFO][main][IgniteKernal]
      
      >>>    __________  ________________
      >>>   /  _/ ___/ |/ /  _/_  __/ __/
      >>>  _/ // (7 7    // /  / / / _/
      >>> /___/\___/_/|_/___/ /_/ /___/
      >>>
      >>> ver. 2.7.0-SNAPSHOT#19700101-sha1:DEV
      >>> 2018 Copyright(C) Apache Software Foundation
      >>>
      >>> Ignite documentation: http://ignite.apache.org
      
      [18:04:22,745][INFO][main][IgniteKernal] Config URL: file:/c:/Work/apache-ignite-2.7.0-SNAPSHOT-bin/server.xml
      [18:04:22,760][INFO][main][IgniteKernal] IgniteConfiguration [igniteInstanceName=null, pubPoolSize=8, svcPoolSize=8, cal
      lbackPoolSize=8, stripedPoolSize=8, sysPoolSize=8, mgmtPoolSize=4, igfsPoolSize=8, dataStreamerPoolSize=8, utilityCacheP
      oolSize=8, utilityCacheKeepAliveTime=60000, p2pPoolSize=2, qryPoolSize=8, igniteHome=c:\Work\apache-ignite-2.7.0-SNAPSHO
      T-bin, igniteWorkDir=c:\Work\apache-ignite-2.7.0-SNAPSHOT-bin\work, mbeanSrv=com.sun.jmx.mbeanserver.JmxMBeanServer@6f94
      fa3e, nodeId=d02069db-6d0b-4a40-b185-1d95fa330853, marsh=BinaryMarshaller [], marshLocJobs=false, daemon=false, p2pEnabl
      ed=false, netTimeout=5000, sndRetryDelay=1000, sndRetryCnt=3, metricsHistSize=10000, metricsUpdateFreq=2000, metricsExpT
      ime=9223372036854775807, discoSpi=TcpDiscoverySpi [addrRslvr=null, sockTimeout=0, ackTimeout=0, marsh=null, reconCnt=10,
       reconDelay=2000, maxAckTimeout=600000, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null], segPlc=ST
      OP, segResolveAttempts=2, waitForSegOnStart=true, allResolversPassReq=true, segChkFreq=10000, commSpi=TcpCommunicationSp
      i [connectGate=null, connPlc=org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$FirstConnectionPolicy@22ff4249,
       enableForcibleNodeKill=false, enableTroubleshootingLog=false, locAddr=null, locHost=null, locPort=47100, locPortRange=1
      00, shmemPort=-1, directBuf=true, directSndBuf=false, idleConnTimeout=600000, connTimeout=5000, maxConnTimeout=600000, r
      econCnt=10, sockSndBuf=32768, sockRcvBuf=32768, msgQueueLimit=0, slowClientQueueLimit=0, nioSrvr=null, shmemSrv=null, us
      ePairedConnections=false, connectionsPerNode=1, tcpNoDelay=true, filterReachableAddresses=false, ackSndThreshold=32, una
      ckedMsgsBufSize=0, sockWriteTimeout=2000, boundTcpPort=-1, boundTcpShmemPort=-1, selectorsCnt=4, selectorSpins=0, addrRs
      lvr=null, ctxInitLatch=java.util.concurrent.CountDownLatch@2d1ef81a[Count = 1], stopping=false], evtSpi=org.apache.ignit
      e.spi.eventstorage.NoopEventStorageSpi@4c402120, colSpi=NoopCollisionSpi [], deploySpi=LocalDeploymentSpi [], indexingSp
      i=org.apache.ignite.spi.indexing.noop.NoopIndexingSpi@815b41f, addrRslvr=null, encryptionSpi=org.apache.ignite.spi.encry
      ption.noop.NoopEncryptionSpi@5542c4ed, clientMode=false, rebalanceThreadPoolSize=1, txCfg=TransactionConfiguration [txSe
      rEnabled=false, dfltIsolation=REPEATABLE_READ, dfltConcurrency=PESSIMISTIC, dfltTxTimeout=0, txTimeoutOnPartitionMapExch
      ange=0, pessimisticTxLogSize=0, pessimisticTxLogLinger=10000, tmLookupClsName=null, txManagerFactory=null, useJtaSync=fa
      lse], cacheSanityCheckEnabled=true, discoStartupDelay=60000, deployMode=SHARED, p2pMissedCacheSize=100, locHost=127.0.0.
      1, timeSrvPortBase=31100, timeSrvPortRange=100, failureDetectionTimeout=10000, sysWorkerBlockedTimeout=null, clientFailu
      reDetectionTimeout=30000, metricsLogFreq=60000, hadoopCfg=null, connectorCfg=ConnectorConfiguration [jettyPath=null, hos
      t=null, port=11211, noDelay=true, directBuf=false, sndBufSize=32768, rcvBufSize=32768, idleQryCurTimeout=600000, idleQry
      CurCheckFreq=60000, sndQueueLimit=0, selectorCnt=4, idleTimeout=7000, sslEnabled=false, sslClientAuth=false, sslCtxFacto
      ry=null, sslFactory=null, portRange=100, threadPoolSize=8, msgInterceptor=null], odbcCfg=null, warmupClos=null, atomicCf
      g=AtomicConfiguration [seqReserveSize=1000, cacheMode=PARTITIONED, backups=1, aff=null, grpName=null], classLdr=null, ss
      lCtxFactory=null, platformCfg=null, binaryCfg=BinaryConfiguration [idMapper=null, nameMapper=null, serializer=null, comp
      actFooter=true], memCfg=null, pstCfg=null, dsCfg=DataStorageConfiguration [sysRegionInitSize=52428800, sysRegionMaxSize=
      104857600, pageSize=0, concLvl=0, dfltDataRegConf=DataRegionConfiguration [name=default, maxSize=104857600, initSize=524
      28800, swapPath=null, pageEvictionMode=DISABLED, evictionThreshold=0.9, emptyPagesPoolSize=100, metricsEnabled=false, me
      tricsSubIntervalCount=5, metricsRateTimeInterval=60000, persistenceEnabled=true, checkpointPageBufSize=0], dataRegions=D
      ataRegionConfiguration[] [DataRegionConfiguration [name=dataRegion1, maxSize=209715200, initSize=104857600, swapPath=nul
      l, pageEvictionMode=RANDOM_LRU, evictionThreshold=0.9, emptyPagesPoolSize=100, metricsEnabled=false, metricsSubIntervalC
      ount=5, metricsRateTimeInterval=60000, persistenceEnabled=true, checkpointPageBufSize=0], DataRegionConfiguration [name=
      dataRegion2, maxSize=104857600, initSize=52428800, swapPath=null, pageEvictionMode=RANDOM_LRU, evictionThreshold=0.9, em
      ptyPagesPoolSize=100, metricsEnabled=false, metricsSubIntervalCount=5, metricsRateTimeInterval=60000, persistenceEnabled
      =true, checkpointPageBufSize=0]], storagePath=null, checkpointFreq=1000, lockWaitTime=10000, checkpointThreads=4, checkp
      ointWriteOrder=SEQUENTIAL, walHistSize=20, maxWalArchiveSize=1073741824, walSegments=10, walSegmentSize=67108864, walPat
      h=db/wal, walArchivePath=db/wal/archive, metricsEnabled=false, walMode=LOG_ONLY, walTlbSize=131072, walBuffSize=0, walFl
      ushFreq=2000, walFsyncDelay=1000, walRecordIterBuffSize=67108864, alwaysWriteFullPages=false, fileIOFactory=org.apache.i
      gnite.internal.processors.cache.persistence.file.AsyncFileIOFactory@3cd3e762, metricsSubIntervalCnt=5, metricsRateTimeIn
      terval=60000, walAutoArchiveAfterInactivity=-1, writeThrottlingEnabled=false, walCompactionEnabled=false, walCompactionL
      evel=1, checkpointReadLockTimeout=null], activeOnStart=true, autoActivation=true, longQryWarnTimeout=3000, sqlConnCfg=nu
      ll, cliConnCfg=ClientConnectorConfiguration [host=null, port=10800, portRange=100, sockSndBufSize=0, sockRcvBufSize=0, t
      cpNoDelay=true, maxOpenCursorsPerConn=128, threadPoolSize=8, idleTimeout=0, jdbcEnabled=true, odbcEnabled=true, thinCliE
      nabled=true, sslEnabled=false, useIgniteSslCtxFactory=true, sslClientAuth=false, sslCtxFactory=null], mvccVacuumThreadCn
      t=2, mvccVacuumFreq=5000, authEnabled=false, failureHnd=null, commFailureRslvr=null]
      [18:04:22,760][INFO][main][IgniteKernal] Daemon mode: off
      [18:04:22,760][INFO][main][IgniteKernal] OS: Windows 10 10.0 amd64
      [18:04:22,760][INFO][main][IgniteKernal] OS user: gridgain
      [18:04:22,760][INFO][main][IgniteKernal] PID: 836
      [18:04:22,760][INFO][main][IgniteKernal] Language runtime: Java Platform API Specification ver. 1.8
      [18:04:22,760][INFO][main][IgniteKernal] VM information: Java(TM) SE Runtime Environment 1.8.0_181-b13 Oracle Corporatio
      n Java HotSpot(TM) 64-Bit Server VM 25.181-b13
      [18:04:22,776][INFO][main][IgniteKernal] VM total memory: 0.96GB
      [18:04:22,776][INFO][main][IgniteKernal] Remote Management [restart: on, REST: on, JMX (remote: on, port: 49148, auth: o
      ff, ssl: off)]
      [18:04:22,776][INFO][main][IgniteKernal] Logger: JavaLogger [quiet=false, config=null]
      [18:04:22,776][INFO][main][IgniteKernal] IGNITE_HOME=c:\Work\apache-ignite-2.7.0-SNAPSHOT-bin
      [18:04:22,776][INFO][main][IgniteKernal] VM arguments: [-XX:+AggressiveOpts, -Xms1g, -Xmx1g, -XX:MaxMetaspaceSize=256m,
      -DIGNITE_QUIET=false, -DIGNITE_SUCCESS_FILE=c:\Work\apache-ignite-2.7.0-SNAPSHOT-bin\work\ignite_success_37805c22-8b1e-4
      14e-86e2-e3af2e5e67df, -Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.port=49148, -Dcom.sun.management.j
      mxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -DIGNITE_HOME=c:\Work\apache-ignite-2.7.0-SNAPSHO
      T-bin, -DIGNITE_PROG_NAME=ignite.bat, -DID=1]
      [18:04:22,776][INFO][main][IgniteKernal] System cache's DataRegion size is configured to 50 MB. Use DataStorageConfigura
      tion.systemRegionInitialSize property to change the setting.
      [18:04:22,776][INFO][main][IgniteKernal] Configured caches [in 'sysMemPlc' dataRegion: ['ignite-sys-cache']]
      [18:04:22,776][WARNING][main][IgniteKernal] Please set system property '-Djava.net.preferIPv4Stack=true' to avoid possib
      le problems in mixed environments.
      [18:04:22,776][WARNING][pub-#19][GridDiagnostic] This operating system has been tested less rigorously: Windows 10 10.0
      amd64. Our team will appreciate the feedback if you experience any problems running ignite in this environment.
      [18:04:22,776][INFO][main][IgniteKernal] 3-rd party licenses can be found at: c:\Work\apache-ignite-2.7.0-SNAPSHOT-bin\l
      ibs\licenses
      [18:04:22,829][INFO][main][IgnitePluginProcessor] Configured plugins:
      [18:04:22,829][INFO][main][IgnitePluginProcessor]   ^-- None
      [18:04:22,829][INFO][main][IgnitePluginProcessor]
      [18:04:22,845][INFO][main][FailureProcessor] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=fals
      e, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED]]]]
      [18:04:22,883][INFO][main][TcpCommunicationSpi] Successfully bound communication NIO server to TCP port [port=47100, loc
      Host=/127.0.0.1, selectorsCnt=4, selectorSpins=0, pairedConn=false]
      [18:04:22,884][WARNING][main][TcpCommunicationSpi] Message queue limit is set to 0 which may lead to potential OOMEs whe
      n running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides
      .
      [18:04:22,901][WARNING][main][NoopCheckpointSpi] Checkpoints are disabled (to enable configure any GridCheckpointSpi imp
      lementation)
      [18:04:22,919][WARNING][main][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon ar
      rival).
      [18:04:22,926][INFO][main][IgniteKernal] Security status [authentication=off, tls/ssl=off]
      [18:04:23,003][INFO][main][CacheObjectBinaryProcessorImpl] Resolved directory for serialized binary metadata: c:\Work\ap
      ache-ignite-2.7.0-SNAPSHOT-bin\work\binary_meta\1
      [18:04:23,125][INFO][main][FilePageStoreManager] Resolved page store work directory: c:\Work\apache-ignite-2.7.0-SNAPSHO
      T-bin\work\db\1
      [18:04:23,127][INFO][main][FileWriteAheadLogManager] Resolved write ahead log work directory: c:\Work\apache-ignite-2.7.
      0-SNAPSHOT-bin\work\db\wal\1
      [18:04:23,128][INFO][main][FileWriteAheadLogManager] Resolved write ahead log archive directory: c:\Work\apache-ignite-2
      .7.0-SNAPSHOT-bin\work\db\wal\archive\1
      [18:04:23,415][INFO][main][FileWriteAheadLogManager] Started write-ahead log manager [mode=LOG_ONLY]
      [18:04:23,415][WARNING][main][GridCacheDatabaseSharedManager] Page eviction mode set for [dataRegion1] data will have no
       effect because the oldest pages are evicted automatically if Ignite persistence is enabled.
      [18:04:23,415][WARNING][main][GridCacheDatabaseSharedManager] Page eviction mode set for [dataRegion2] data will have no
       effect because the oldest pages are evicted automatically if Ignite persistence is enabled.
      [18:04:23,415][WARNING][main][GridCacheDatabaseSharedManager] DataRegionConfiguration.maxWalArchiveSize instead DataRegi
      onConfiguration.walHistorySize would be used for removing old archive wal files
      [18:04:23,462][INFO][main][PartitionsEvictManager] Evict partition permits=2
      [18:04:23,631][INFO][main][ClientListenerProcessor] Client connector processor has started on TCP port 10800
      [18:04:23,663][INFO][main][GridTcpRestProtocol] Command protocol successfully started [name=TCP binary, host=/127.0.0.1,
       port=11211]
      [18:04:23,763][INFO][main][IgniteKernal] Non-loopback local IPs: 172.25.4.52, fe80:0:0:0:2d2b:5bfb:e052:9fb0%eth5
      [18:04:23,763][INFO][main][IgniteKernal] Enabled local MACs: 54A050520B61
      [18:04:23,763][INFO][main][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null]
      [18:04:23,779][INFO][main][PageMemoryImpl] Started page memory [memoryAllocated=100,0 MiB, pages=24808, tableSize=1,9 Mi
      B, checkpointBuffer=100,0 MiB]
      [18:04:23,779][INFO][main][GridCacheDatabaseSharedManager] Checking memory state [lastValidPos=FileWALPointer [idx=0, fi
      leOff=0, len=0], lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000
      000]
      [18:04:23,794][INFO][main][GridCacheDatabaseSharedManager] Applying lost cache updates since last checkpoint record [las
      tMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000]
      [18:04:23,794][INFO][main][GridCacheDatabaseSharedManager] Finished applying WAL changes [updatesApplied=0, time=0ms]
      [18:04:23,816][INFO][main][TcpDiscoverySpi] Connection check threshold is calculated: 10000
      [18:04:23,832][INFO][main][TcpDiscoverySpi] Successfully bound to TCP port [port=47500, localHost=127.0.0.1/127.0.0.1, l
      ocNodeId=d02069db-6d0b-4a40-b185-1d95fa330853]
      [18:04:32,790][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/127.0.0.1,
       rmtPort=61618]
      [18:04:32,822][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/12
      7.0.0.1, rmtPort=61618]
      [18:04:32,822][INFO][tcp-disco-sock-reader-#4][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/127.0.0
      .1:61618, rmtPort=61618]
      [18:04:32,839][INFO][tcp-disco-sock-reader-#4][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/127.0.
      0.1:61618, rmtPort=61618
      [18:04:43,227][INFO][disco-notifier-worker-#41][MvccProcessorImpl] Assigned mvcc coordinator [crd=MvccCoordinator [nodeI
      d=d02069db-6d0b-4a40-b185-1d95fa330853, crdVer=1540479883228, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=0]],
       crdNode=TcpDiscoveryNode [id=d02069db-6d0b-4a40-b185-1d95fa330853, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127
      .0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540479863832, loc=true, ver=2.7.0#19700101-sha1:00
      000000, isClient=false]]
      [18:04:43,310][INFO][main][IgniteKernal] Performance suggestions for grid  (fix if possible)
      [18:04:43,310][INFO][main][IgniteKernal] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
      [18:04:43,310][INFO][main][IgniteKernal]   ^-- Enable G1 Garbage Collector (add '-XX:+UseG1GC' to JVM options)
      [18:04:43,310][INFO][main][IgniteKernal]   ^-- Set max direct memory size if getting 'OOME: Direct buffer memory' (add '
      -XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM options)
      [18:04:43,310][INFO][main][IgniteKernal]   ^-- Disable processing of calls to System.gc() (add '-XX:+DisableExplicitGC'
      to JVM options)
      [18:04:43,310][INFO][main][IgniteKernal] Refer to this page for more performance suggestions: https://apacheignite.readm
      e.io/docs/jvm-and-system-tuning
      [18:04:43,310][INFO][main][IgniteKernal]
      [18:04:43,310][INFO][main][IgniteKernal] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
      [18:04:43,310][INFO][main][IgniteKernal] Data Regions Configured:
      [18:04:43,310][INFO][main][IgniteKernal]   ^-- default [initSize=50,0 MiB, maxSize=100,0 MiB, persistence=true]
      [18:04:43,310][INFO][main][IgniteKernal]   ^-- dataRegion1 [initSize=100,0 MiB, maxSize=200,0 MiB, persistence=true]
      [18:04:43,310][INFO][main][IgniteKernal]   ^-- dataRegion2 [initSize=50,0 MiB, maxSize=100,0 MiB, persistence=true]
      [18:04:43,310][INFO][main][IgniteKernal]
      
      >>> +------------------------------------------+
      >>> Ignite ver. 2.7.0-SNAPSHOT#19700101-sha1:DEV
      >>> +------------------------------------------+
      >>> OS name: Windows 10 10.0 amd64
      >>> CPU(s): 8
      >>> Heap: 1.0GB
      >>> VM name: 836@work-pc
      >>> Local node [ID=D02069DB-6D0B-4A40-B185-1D95FA330853, order=1, clientMode=false]
      >>> Local node addresses: [127.0.0.1]
      >>> Local ports: TCP:10800 TCP:11211 TCP:47100 TCP:47500
      
      [18:04:43,310][INFO][main][IgniteKernal] >>> Ignite cluster is not active (limited functionality available). Use control
      .(sh|bat) script or IgniteCluster interface to activate.
      [18:04:43,310][INFO][main][GridDiscoveryManager] Topology snapshot [ver=1, locNode=d02069db, servers=1, clients=0, state
      =INACTIVE, CPUs=8, offheap=0.39GB, heap=1.0GB]
      [18:04:43,341][INFO][rest-#45][GridClusterStateProcessor] Sending activate request with BaselineTopology BaselineTopolog
      y [id=0, branchingHash=49, branchingType='New BaselineTopology', baselineNodes=[1]]
      [18:04:43,357][INFO][disco-notifier-worker-#41][GridClusterStateProcessor] Received activate request with BaselineTopolo
      gy[id=0]
      [18:04:43,357][INFO][disco-notifier-worker-#41][GridClusterStateProcessor] Started state transition: true
      [18:04:43,357][INFO][exchange-worker-#43][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTo
      pVer=1], mvccCrd=MvccCoordinator [nodeId=d02069db-6d0b-4a40-b185-1d95fa330853, crdVer=1540479883228, topVer=AffinityTopo
      logyVersion [topVer=1, minorTopVer=0]], mvccCrdChange=true, crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=d02069db-6d0b-4a
      40-b185-1d95fa330853, customEvt=ChangeGlobalStateMessage [id=a6641cba661-606014c4-c468-4812-86de-7e8ae4cc30b9, reqId=a2c
      c8ce1-aff1-4600-aa51-a600474de991, initiatingNodeId=d02069db-6d0b-4a40-b185-1d95fa330853, activate=true, baselineTopolog
      y=BaselineTopology [id=0, branchingHash=49, branchingType='New BaselineTopology', baselineNodes=[1]], forceChangeBaselin
      eTopology=false, timestamp=1540479883341], allowMerge=false]
      [18:04:43,357][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Start activation process [nodeId=d02069db-6d0
      b-4a40-b185-1d95fa330853, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
      [18:04:43,357][INFO][exchange-worker-#43][FilePageStoreManager] Resolved page store work directory: c:\Work\apache-ignit
      e-2.7.0-SNAPSHOT-bin\work\db\1
      [18:04:43,357][INFO][exchange-worker-#43][FileWriteAheadLogManager] Resolved write ahead log work directory: c:\Work\apa
      che-ignite-2.7.0-SNAPSHOT-bin\work\db\wal\1
      [18:04:43,372][INFO][exchange-worker-#43][FileWriteAheadLogManager] Resolved write ahead log archive directory: c:\Work\
      apache-ignite-2.7.0-SNAPSHOT-bin\work\db\wal\archive\1
      [18:04:43,372][INFO][exchange-worker-#43][FileWriteAheadLogManager] Started write-ahead log manager [mode=LOG_ONLY]
      [18:04:43,379][INFO][exchange-worker-#43][PageMemoryImpl] Started page memory [memoryAllocated=200,0 MiB, pages=49624, t
      ableSize=3,9 MiB, checkpointBuffer=200,0 MiB]
      [18:04:43,379][INFO][exchange-worker-#43][PageMemoryImpl] Started page memory [memoryAllocated=100,0 MiB, pages=24808, t
      ableSize=1,9 MiB, checkpointBuffer=100,0 MiB]
      [18:04:43,379][INFO][exchange-worker-#43][PageMemoryImpl] Started page memory [memoryAllocated=100,0 MiB, pages=24808, t
      ableSize=1,9 MiB, checkpointBuffer=100,0 MiB]
      [18:04:43,379][INFO][exchange-worker-#43][PageMemoryImpl] Started page memory [memoryAllocated=100,0 MiB, pages=24808, t
      ableSize=1,9 MiB, checkpointBuffer=100,0 MiB]
      [18:04:43,379][INFO][exchange-worker-#43][PageMemoryImpl] Started page memory [memoryAllocated=100,0 MiB, pages=24808, t
      ableSize=1,9 MiB, checkpointBuffer=100,0 MiB]
      [18:04:43,379][INFO][exchange-worker-#43][PageMemoryImpl] Started page memory [memoryAllocated=100,0 MiB, pages=24808, t
      ableSize=1,9 MiB, checkpointBuffer=100,0 MiB]
      [18:04:43,395][INFO][exchange-worker-#43][msg] Components activation performed in 38 ms.
      [18:04:43,410][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endM
      arker=null]
      [18:04:43,410][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Checking memory state [lastValidPos=FileWALPoi
      nter [idx=0, fileOff=0, len=0], lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000
      -0000-000000000000]
      [18:04:43,410][INFO][exchange-worker-#43][FileWriteAheadLogManager] Resuming logging to WAL segment [file=c:\Work\apache
      -ignite-2.7.0-SNAPSHOT-bin\work\db\wal\1\0000000000000000.wal, offset=0, ver=2]
      [18:04:43,457][INFO][exchange-worker-#43][GridClusterStateProcessor] Writing BaselineTopology[id=0]
      [18:04:43,473][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Binary recovery performed in 78 ms.
      [18:04:43,473][INFO][exchange-worker-#43][CacheAffinitySharedManager] Updating caches registry performed in 0 ms.
      [18:04:43,611][INFO][exchange-worker-#43][GridCacheProcessor] Started cache [name=ignite-sys-cache, id=-2100569601, memo
      ryPolicyName=sysMemPlc, mode=REPLICATED, atomicity=TRANSACTIONAL, backups=2147483647, mvcc=false], encryptionEnabled=fal
      se]
      [18:04:43,611][INFO][exchange-worker-#43][CacheAffinitySharedManager] Caches starting performed in 138 ms.
      [18:04:43,611][INFO][exchange-worker-#43][CacheAffinitySharedManager] Affinity initialization for started caches perform
      ed in 0 ms.
      [18:04:43,611][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Successfully activated caches [nodeId=d02069d
      b-6d0b-4a40-b185-1d95fa330853, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
      [18:04:43,626][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=LOCAL]
      [18:04:43,626][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endM
      arker=null]
      [18:04:43,626][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Applying lost cache updates since last checkpo
      int record [lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000]
      [18:04:43,679][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Restoring partition state for local groups [cn
      tPartStateWal=0, lastCheckpointId=00000000-0000-0000-0000-000000000000]
      [18:04:43,695][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Finished restoring partition state for local g
      roups [cntProcessed=100, cntPartStateWal=0, time=16ms]
      [18:04:43,695][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Finished applying WAL changes [updatesApplied=
      0, time=69ms]
      [18:04:43,695][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Logical recovery performed in 69 ms.
      [18:04:43,726][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Checkpointer initilialzation performed in 15 m
      s.
      [18:04:43,726][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] finishExchangeOnCoordinator [topVer=AffinityT
      opologyVersion [topVer=1, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
      [18:04:43,726][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Affinity changes (coordinator) applied in 0 m
      s.
      [18:04:43,726][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Partitions validation performed in 0 ms.
      [18:04:43,726][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Partitions assignment performed in 0 ms.
      [18:04:43,726][INFO][exchange-worker-#43][CacheAffinitySharedManager] Affinity recalculation (custom message) performed
      in 0 ms.
      [18:04:43,742][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Preparing Full Message performed in 16 ms.
      [18:04:43,742][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Sending Full Message to all nodes performed i
      n 0 ms.
      [18:04:43,742][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopo
      logyVersion [topVer=1, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], err=null]
      [18:04:43,742][INFO][disco-notifier-worker-#41][GridClusterStateProcessor] Received state change finish message: true
      [18:04:43,742][INFO][exchange-worker-#43][GridCachePartitionExchangeManager] Sending Full Message for AffinityTopologyVe
      rsion [topVer=1, minorTopVer=1] performed in 0 ms.
      [18:04:43,742][INFO][exchange-worker-#43][MvccProcessorImpl] Initialize local node as mvcc coordinator [node=d02069db-6d
      0b-4a40-b185-1d95fa330853, crdVer=1540479883228]
      [18:04:43,742][INFO][exchange-worker-#43][GridCacheProcessor] Finish proxy initialization, cacheName=ignite-sys-cache, l
      ocalNodeId=d02069db-6d0b-4a40-b185-1d95fa330853
      [18:04:43,758][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Completed partition exchange [localNode=d0206
      9db-6d0b-4a40-b185-1d95fa330853, exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=1, min
      orTopVer=1], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode [id=d02069db-6d0b-4a40-b185-1d95fa330853, addrs=ArrayLis
      t [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540479863832
      , loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], done=true], topVer=AffinityTopologyVersion [topVer=1, min
      orTopVer=1], durationFromInit=401]
      [18:04:43,758][INFO][exchange-worker-#43][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=1, minorT
      opVer=1], crd=true]
      [18:04:43,758][INFO][exchange-worker-#43][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [t
      op=AffinityTopologyVersion [topVer=1, minorTopVer=1], force=false, evt=DISCOVERY_CUSTOM_EVT, node=d02069db-6d0b-4a40-b18
      5-1d95fa330853]
      [18:04:43,758][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Checkpoint started [checkpointId=143137c0
      -86bf-44a0-9f9c-efba0161e74d, startPtr=FileWALPointer [idx=0, fileOff=119489, len=1947], checkpointLockWait=0ms, checkpo
      intLockHoldTime=15ms, walCpRecordFsyncDuration=16ms, pages=17, reason='timeout']
      [18:04:43,773][INFO][sys-#54][GridClusterStateProcessor] Successfully performed final activation steps [nodeId=d02069db-
      6d0b-4a40-b185-1d95fa330853, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
      [18:04:43,795][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Checkpoint finished [cpId=143137c0-86bf-4
      4a0-9f9c-efba0161e74d, pages=17, markPos=FileWALPointer [idx=0, fileOff=119489, len=1947], walSegmentsCleared=0, walSegm
      entsCovered=[], markDuration=62ms, pagesWrite=0ms, fsync=22ms, total=84ms]
      [18:04:44,729][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modifi
      ed) [checkpointLockWait=0ms, checkpointLockHoldTime=0ms, reason='timeout']
      [18:04:48,037][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/127.0.0.1,
       rmtPort=61665]
      [18:04:48,037][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/12
      7.0.0.1, rmtPort=61665]
      [18:04:48,041][INFO][tcp-disco-sock-reader-#5][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/127.0.0
      .1:61665, rmtPort=61665]
      [18:04:48,063][INFO][tcp-disco-sock-reader-#5][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/127.0.
      0.1:61665, rmtPort=61665
      [18:04:48,069][INFO][tcp-disco-msg-worker-#2][GridEncryptionManager] Joining node doesn't have encryption data [node=5df
      37010-aaf1-4c1c-88fb-53b8b178c4f5]
      [18:04:48,093][INFO][tcp-disco-msg-worker-#2][TcpDiscoverySpi] New next node [newNext=TcpDiscoveryNode [id=5df37010-aaf1
      -4c1c-88fb-53b8b178c4f5, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47501], discPort=47501, order=0, int
      Order=2, lastExchangeTime=1540479888049, loc=false, ver=2.7.0#19700101-sha1:00000000, isClient=false]]
      [18:04:48,129][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/127.0.0.1,
       rmtPort=61667]
      [18:04:48,129][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/12
      7.0.0.1, rmtPort=61667]
      [18:04:48,130][INFO][tcp-disco-sock-reader-#6][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/127.0.0
      .1:61667, rmtPort=61667]
      [18:04:48,136][INFO][disco-event-worker-#42][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=5df3
      7010-aaf1-4c1c-88fb-53b8b178c4f5, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47501], discPort=47501, ord
      er=2, intOrder=2, lastExchangeTime=1540479888049, loc=false, ver=2.7.0#19700101-sha1:00000000, isClient=false]
      [18:04:48,137][INFO][disco-event-worker-#42][GridDiscoveryManager] Topology snapshot [ver=2, locNode=d02069db, servers=2
      , clients=0, state=ACTIVE, CPUs=8, offheap=0.78GB, heap=2.0GB]
      [18:04:48,137][INFO][disco-event-worker-#42][GridDiscoveryManager]   ^-- Baseline [id=0, size=1, online=1, offline=0]
      [18:04:48,138][INFO][exchange-worker-#43][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=2, minorTo
      pVer=0], mvccCrd=MvccCoordinator [nodeId=d02069db-6d0b-4a40-b185-1d95fa330853, crdVer=1540479883228, topVer=AffinityTopo
      logyVersion [topVer=1, minorTopVer=0]], mvccCrdChange=false, crd=true, evt=NODE_JOINED, evtNode=5df37010-aaf1-4c1c-88fb-
      53b8b178c4f5, customEvt=null, allowMerge=true]
      [18:04:48,145][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0], waitTime=0ms, futInfo=NA, mode=DISTRIBUTED]
      [18:04:48,148][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partitions release latch
      : ServerLatch [permits=0, pendingAcks=HashSet [], super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion [t
      opVer=2, minorTopVer=0]]]
      [18:04:48,148][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0], waitTime=0ms, futInfo=NA, mode=LOCAL]
      [18:04:48,148][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Logical recovery performed in 0 ms.
      [18:04:48,149][INFO][exchange-worker-#43][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=2, minorT
      opVer=0], crd=true]
      [18:04:48,725][INFO][grid-nio-worker-tcp-comm-0-#24][TcpCommunicationSpi] Accepted incoming communication connection [lo
      cAddr=/127.0.0.1:47100, rmtAddr=/127.0.0.1:61669]
      [18:04:48,741][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Coordinator received single message [ver=AffinityTopology
      Version [topVer=2, minorTopVer=0], node=5df37010-aaf1-4c1c-88fb-53b8b178c4f5, allReceived=true]
      [18:04:48,741][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Coordinator received all messages, try merge [ver=Affinit
      yTopologyVersion [topVer=2, minorTopVer=0]]
      [18:04:48,741][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Exchanges merging performed in 0 ms.
      [18:04:48,741][INFO][sys-#56][GridDhtPartitionsExchangeFuture] finishExchangeOnCoordinator [topVer=AffinityTopologyVersi
      on [topVer=2, minorTopVer=0], resVer=AffinityTopologyVersion [topVer=2, minorTopVer=0]]
      [18:04:48,741][INFO][sys-#56][CacheAffinitySharedManager] Affinity recalculation (on server join) performed in 0 ms.
      [18:04:48,756][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Affinity changes (coordinator) applied in 15 ms.
      [18:04:48,756][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Partitions validation performed in 0 ms.
      [18:04:48,756][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Partitions assignment performed in 0 ms.
      [18:04:48,756][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Preparing Full Message performed in 0 ms.
      [18:04:48,756][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Sending Full Message performed in 0 ms.
      [18:04:48,756][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Sending Full Message to all nodes performed in 0 ms.
      [18:04:48,756][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion
      [topVer=2, minorTopVer=0], resVer=AffinityTopologyVersion [topVer=2, minorTopVer=0], err=null]
      [18:04:48,756][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Detecting lost partitions performed in 0 ms.
      [18:04:48,756][INFO][sys-#56][GridDhtPartitionsExchangeFuture] Completed partition exchange [localNode=d02069db-6d0b-4a4
      0-b185-1d95fa330853, exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0],
       evt=NODE_JOINED, evtNode=TcpDiscoveryNode [id=5df37010-aaf1-4c1c-88fb-53b8b178c4f5, addrs=ArrayList [127.0.0.1], sockAd
      drs=HashSet [/127.0.0.1:47501], discPort=47501, order=2, intOrder=2, lastExchangeTime=1540479888049, loc=false, ver=2.7.
      0#19700101-sha1:00000000, isClient=false], done=true], topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0], duratio
      nFromInit=620]
      [18:04:48,756][INFO][exchange-worker-#43][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [t
      op=AffinityTopologyVersion [topVer=2, minorTopVer=0], force=false, evt=NODE_JOINED, node=5df37010-aaf1-4c1c-88fb-53b8b17
      8c4f5]
      [18:04:49,769][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modifi
      ed) [checkpointLockWait=0ms, checkpointLockHoldTime=3ms, reason='timeout']
      [18:04:55,817][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modifi
      ed) [checkpointLockWait=0ms, checkpointLockHoldTime=1ms, reason='timeout']
      [18:05:02,884][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modifi
      ed) [checkpointLockWait=0ms, checkpointLockHoldTime=2ms, reason='timeout']
      [18:05:09,804][INFO][exchange-worker-#43][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=2, minorTo
      pVer=1], mvccCrd=MvccCoordinator [nodeId=d02069db-6d0b-4a40-b185-1d95fa330853, crdVer=1540479883228, topVer=AffinityTopo
      logyVersion [topVer=1, minorTopVer=0]], mvccCrdChange=false, crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=d02069db-6d0b-4
      a40-b185-1d95fa330853, customEvt=DynamicCacheChangeBatch [id=c6741cba661-606014c4-c468-4812-86de-7e8ae4cc30b9, reqs=Arra
      yList [DynamicCacheChangeRequest [cacheName=SQL_PUBLIC_T1, hasCfg=true, nodeId=d02069db-6d0b-4a40-b185-1d95fa330853, cli
      entStartOnly=false, stop=false, destroy=false, disabledAfterStartfalse]], exchangeActions=ExchangeActions [startCaches=[
      SQL_PUBLIC_T1], stopCaches=null, startGrps=[SQL_PUBLIC_T1], stopGrps=[], resetParts=null, stateChangeRequest=null], star
      tCaches=false], allowMerge=false]
      [18:05:09,804][INFO][exchange-worker-#43][CacheAffinitySharedManager] Updating caches registry performed in 0 ms.
      [18:05:09,875][INFO][exchange-worker-#43][MvccProcessorImpl] Mvcc processor started.
      [18:05:09,912][INFO][exchange-worker-#43][GridCacheProcessor] Started cache [name=SQL_PUBLIC_T1, id=-1691533662, memoryP
      olicyName=default, mode=PARTITIONED, atomicity=TRANSACTIONAL_SNAPSHOT, backups=1, mvcc=true], encryptionEnabled=false]
      [18:05:09,912][INFO][exchange-worker-#43][CacheAffinitySharedManager] Caches starting performed in 108 ms.
      [18:05:09,915][INFO][exchange-worker-#43][GridAffinityAssignmentCache] Local node affinity assignment distribution is no
      t ideal [cache=SQL_PUBLIC_T1, expectedPrimary=1024,00, actualPrimary=1024, expectedBackups=1024,00, actualBackups=0, war
      ningThreshold=50,00%]
      [18:05:09,916][INFO][exchange-worker-#43][CacheAffinitySharedManager] Affinity initialization for started caches perform
      ed in 3 ms.
      [18:05:09,917][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=2, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=DISTRIBUTED]
      [18:05:09,934][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Checkpoint started [checkpointId=39c009dc
      -7bd4-465e-8a4d-156ea670668b, startPtr=FileWALPointer [idx=0, fileOff=131199, len=1953], checkpointLockWait=5ms, checkpo
      intLockHoldTime=1ms, walCpRecordFsyncDuration=9ms, pages=13, reason='timeout']
      [18:05:09,945][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partitions release latch
      : ServerLatch [permits=0, pendingAcks=HashSet [], super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion [t
      opVer=2, minorTopVer=1]]]
      [18:05:09,946][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=2, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=LOCAL]
      [18:05:09,947][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Checkpoint finished [cpId=39c009dc-7bd4-4
      65e-8a4d-156ea670668b, pages=13, markPos=FileWALPointer [idx=0, fileOff=131199, len=1953], walSegmentsCleared=0, walSegm
      entsCovered=[], markDuration=22ms, pagesWrite=2ms, fsync=11ms, total=40ms]
      [18:05:09,963][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Logical recovery performed in 17 ms.
      [18:05:10,014][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Coordinator received single message [ver=Affi
      nityTopologyVersion [topVer=2, minorTopVer=1], node=5df37010-aaf1-4c1c-88fb-53b8b178c4f5, allReceived=true]
      [18:05:10,014][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] finishExchangeOnCoordinator [topVer=AffinityT
      opologyVersion [topVer=2, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=2, minorTopVer=1]]
      [18:05:10,015][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Affinity changes (coordinator) applied in 0 m
      s.
      [18:05:10,016][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Partitions validation performed in 1 ms.
      [18:05:10,020][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Partitions assignment performed in 4 ms.
      [18:05:10,025][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Preparing Full Message performed in 4 ms.
      [18:05:10,025][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Sending Full Message performed in 0 ms.
      [18:05:10,025][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Sending Full Message to all nodes performed i
      n 0 ms.
      [18:05:10,026][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopo
      logyVersion [topVer=2, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=2, minorTopVer=1], err=null]
      [18:05:10,029][INFO][exchange-worker-#43][GridCacheProcessor] Finish proxy initialization, cacheName=SQL_PUBLIC_T1, loca
      lNodeId=d02069db-6d0b-4a40-b185-1d95fa330853
      [18:05:10,030][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Completed partition exchange [localNode=d0206
      9db-6d0b-4a40-b185-1d95fa330853, exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=2, min
      orTopVer=1], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode [id=d02069db-6d0b-4a40-b185-1d95fa330853, addrs=ArrayLis
      t [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540479909804
      , loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], done=true], topVer=AffinityTopologyVersion [topVer=2, min
      orTopVer=1], durationFromInit=222]
      [18:05:10,030][INFO][exchange-worker-#43][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=2, minorT
      opVer=1], crd=true]
      [18:05:10,031][INFO][pub-#70][GridCacheDatabaseSharedManager] Finished indexes rebuilding for cache [name=SQL_PUBLIC_T1,
       grpName=null]
      [18:05:10,033][INFO][exchange-worker-#43][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [t
      op=AffinityTopologyVersion [topVer=2, minorTopVer=1], force=false, evt=DISCOVERY_CUSTOM_EVT, node=d02069db-6d0b-4a40-b18
      5-1d95fa330853]
      [18:05:10,919][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modifi
      ed) [checkpointLockWait=0ms, checkpointLockHoldTime=4ms, reason='timeout']
      [18:05:14,456][INFO][disco-event-worker-#42][GridDiscoveryManager] Node left topology: TcpDiscoveryNode [id=5df37010-aaf
      1-4c1c-88fb-53b8b178c4f5, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47501], discPort=47501, order=2, in
      tOrder=2, lastExchangeTime=1540479888049, loc=false, ver=2.7.0#19700101-sha1:00000000, isClient=false]
      [18:05:14,456][INFO][disco-event-worker-#42][GridDiscoveryManager] Topology snapshot [ver=3, locNode=d02069db, servers=1
      , clients=0, state=ACTIVE, CPUs=8, offheap=0.39GB, heap=1.0GB]
      [18:05:14,456][INFO][disco-event-worker-#42][GridDiscoveryManager]   ^-- Baseline [id=0, size=1, online=1, offline=0]
      [18:05:14,488][INFO][exchange-worker-#43][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=3, minorTo
      pVer=0], mvccCrd=MvccCoordinator [nodeId=d02069db-6d0b-4a40-b185-1d95fa330853, crdVer=1540479883228, topVer=AffinityTopo
      logyVersion [topVer=1, minorTopVer=0]], mvccCrdChange=false, crd=true, evt=NODE_LEFT, evtNode=5df37010-aaf1-4c1c-88fb-53
      b8b178c4f5, customEvt=null, allowMerge=true]
      [18:05:14,488][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=3, minorTopVer=0], waitTime=0ms, futInfo=NA, mode=DISTRIBUTED]
      [18:05:14,488][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partitions release latch
      : ServerLatch [permits=0, pendingAcks=HashSet [], super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion [t
      opVer=3, minorTopVer=0]]]
      [18:05:14,488][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=3, minorTopVer=0], waitTime=0ms, futInfo=NA, mode=LOCAL]
      [18:05:14,488][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Logical recovery performed in 0 ms.
      [18:05:14,488][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Coordinator received all messages, try merge
      [ver=AffinityTopologyVersion [topVer=3, minorTopVer=0]]
      [18:05:14,488][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Exchanges merging performed in 0 ms.
      [18:05:14,488][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] finishExchangeOnCoordinator [topVer=AffinityT
      opologyVersion [topVer=3, minorTopVer=0], resVer=AffinityTopologyVersion [topVer=3, minorTopVer=0]]
      [18:05:14,488][INFO][exchange-worker-#43][GridAffinityAssignmentCache] Local node affinity assignment distribution is no
      t ideal [cache=SQL_PUBLIC_T1, expectedPrimary=1024,00, actualPrimary=1024, expectedBackups=1024,00, actualBackups=0, war
      ningThreshold=50,00%]
      [18:05:14,511][INFO][exchange-worker-#43][CacheAffinitySharedManager] Affinity recalculation (on server left) performed
      in 23 ms.
      [18:05:14,513][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Affinity changes (coordinator) applied in 25
      ms.
      [18:05:14,513][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Partitions validation performed in 0 ms.
      [18:05:14,513][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Detecting lost partitions performed in 0 ms.
      [18:05:14,513][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Preparing Full Message performed in 0 ms.
      [18:05:14,513][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Sending Full Message to all nodes performed i
      n 0 ms.
      [18:05:14,513][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopo
      logyVersion [topVer=3, minorTopVer=0], resVer=AffinityTopologyVersion [topVer=3, minorTopVer=0], err=null]
      [18:05:14,513][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Detecting lost partitions performed in 0 ms.
      [18:05:14,513][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Completed partition exchange [localNode=d0206
      9db-6d0b-4a40-b185-1d95fa330853, exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=3, min
      orTopVer=0], evt=NODE_LEFT, evtNode=TcpDiscoveryNode [id=5df37010-aaf1-4c1c-88fb-53b8b178c4f5, addrs=ArrayList [127.0.0.
      1], sockAddrs=HashSet [/127.0.0.1:47501], discPort=47501, order=2, intOrder=2, lastExchangeTime=1540479888049, loc=false
      , ver=2.7.0#19700101-sha1:00000000, isClient=false], done=true], topVer=AffinityTopologyVersion [topVer=3, minorTopVer=0
      ], durationFromInit=15]
      [18:05:14,513][INFO][exchange-worker-#43][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=3, minorT
      opVer=0], crd=true]
      [18:05:14,513][INFO][exchange-worker-#43][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [t
      op=AffinityTopologyVersion [topVer=3, minorTopVer=0], force=false, evt=NODE_LEFT, node=5df37010-aaf1-4c1c-88fb-53b8b178c
      4f5]
      [18:05:14,982][INFO][tcp-disco-sock-reader-#6][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/127.0.
      0.1:61667, rmtPort=61667
      [18:05:21,967][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modifi
      ed) [checkpointLockWait=0ms, checkpointLockHoldTime=5ms, reason='timeout']
      [18:05:23,818][INFO][exchange-worker-#43][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=3, minorTo
      pVer=1], mvccCrd=MvccCoordinator [nodeId=d02069db-6d0b-4a40-b185-1d95fa330853, crdVer=1540479883228, topVer=AffinityTopo
      logyVersion [topVer=1, minorTopVer=0]], mvccCrdChange=false, crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=d02069db-6d0b-4
      a40-b185-1d95fa330853, customEvt=DynamicCacheChangeBatch [id=ea741cba661-606014c4-c468-4812-86de-7e8ae4cc30b9, reqs=Arra
      yList [DynamicCacheChangeRequest [cacheName=SQL_PUBLIC_T1, hasCfg=false, nodeId=d02069db-6d0b-4a40-b185-1d95fa330853, cl
      ientStartOnly=false, stop=true, destroy=false, disabledAfterStartfalse]], exchangeActions=ExchangeActions [startCaches=n
      ull, stopCaches=[SQL_PUBLIC_T1], startGrps=[], stopGrps=[SQL_PUBLIC_T1, destroy=true], resetParts=null, stateChangeReque
      st=null], startCaches=false], allowMerge=false]
      [18:05:23,819][INFO][exchange-worker-#43][CacheAffinitySharedManager] Updating caches registry performed in 1 ms.
      [18:05:23,820][INFO][exchange-worker-#43][CacheAffinitySharedManager] Caches starting performed in 0 ms.
      [18:05:23,820][INFO][exchange-worker-#43][CacheAffinitySharedManager] Affinity initialization for started caches perform
      ed in 0 ms.
      [18:05:23,820][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=DISTRIBUTED]
      [18:05:23,821][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partitions release latch
      : ServerLatch [permits=0, pendingAcks=HashSet [], super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion [t
      opVer=3, minorTopVer=1]]]
      [18:05:23,821][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=LOCAL]
      [18:05:23,821][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Logical recovery performed in 0 ms.
      [18:05:23,821][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] finishExchangeOnCoordinator [topVer=AffinityT
      opologyVersion [topVer=3, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=3, minorTopVer=1]]
      [18:05:23,821][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Affinity changes (coordinator) applied in 0 m
      s.
      [18:05:23,821][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Partitions validation performed in 0 ms.
      [18:05:23,822][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Partitions assignment performed in 0 ms.
      [18:05:23,825][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Preparing Full Message performed in 3 ms.
      [18:05:23,825][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Sending Full Message to all nodes performed i
      n 0 ms.
      [18:05:23,825][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopo
      logyVersion [topVer=3, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=3, minorTopVer=1], err=null]
      [18:05:23,827][INFO][db-checkpoint-thread-#49][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modifi
      ed) [checkpointLockWait=0ms, checkpointLockHoldTime=2ms, reason='caches stop']
      [18:05:23,833][INFO][exchange-worker-#43][GridCacheProcessor] Stopped cache [cacheName=SQL_PUBLIC_T1]
      [18:05:23,851][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Completed partition exchange [localNode=d0206
      9db-6d0b-4a40-b185-1d95fa330853, exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=3, min
      orTopVer=1], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode [id=d02069db-6d0b-4a40-b185-1d95fa330853, addrs=ArrayLis
      t [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540479914456
      , loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], done=true], topVer=AffinityTopologyVersion [topVer=3, min
      orTopVer=1], durationFromInit=33]
      [18:05:23,851][INFO][exchange-worker-#43][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=3, minorT
      opVer=1], crd=true]
      [18:05:23,853][INFO][exchange-worker-#43][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [t
      op=AffinityTopologyVersion [topVer=3, minorTopVer=1], force=false, evt=DISCOVERY_CUSTOM_EVT, node=d02069db-6d0b-4a40-b18
      5-1d95fa330853]
      [18:05:36,638][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/127.0.0.1,
       rmtPort=61721]
      [18:05:36,638][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/12
      7.0.0.1, rmtPort=61721]
      [18:05:36,638][INFO][tcp-disco-sock-reader-#7][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/127.0.0
      .1:61721, rmtPort=61721]
      [18:05:36,669][INFO][tcp-disco-sock-reader-#7][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/127.0.
      0.1:61721, rmtPort=61721
      [18:05:36,685][INFO][tcp-disco-msg-worker-#2][GridEncryptionManager] Joining node doesn't have encryption data [node=d07
      9c11d-bd7d-4d65-93ab-dee8d0b280e3]
      [18:05:36,685][INFO][tcp-disco-msg-worker-#2][TcpDiscoverySpi] New next node [newNext=TcpDiscoveryNode [id=d079c11d-bd7d
      -4d65-93ab-dee8d0b280e3, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47501], discPort=47501, order=0, int
      Order=3, lastExchangeTime=1540479936669, loc=false, ver=2.7.0#19700101-sha1:00000000, isClient=false]]
      [18:05:36,738][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/127.0.0.1,
       rmtPort=61723]
      [18:05:36,738][INFO][tcp-disco-srvr-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/12
      7.0.0.1, rmtPort=61723]
      [18:05:36,738][INFO][tcp-disco-sock-reader-#8][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/127.0.0
      .1:61723, rmtPort=61723]
      [18:05:36,754][INFO][disco-event-worker-#42][GridDiscoveryManager] Added new node to topology: TcpDiscoveryNode [id=d079
      c11d-bd7d-4d65-93ab-dee8d0b280e3, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47501], discPort=47501, ord
      er=4, intOrder=3, lastExchangeTime=1540479936669, loc=false, ver=2.7.0#19700101-sha1:00000000, isClient=false]
      [18:05:36,754][INFO][disco-event-worker-#42][GridDiscoveryManager] Topology snapshot [ver=4, locNode=d02069db, servers=2
      , clients=0, state=ACTIVE, CPUs=8, offheap=0.78GB, heap=2.0GB]
      [18:05:36,754][INFO][disco-event-worker-#42][GridDiscoveryManager]   ^-- Baseline [id=0, size=1, online=1, offline=0]
      [18:05:36,754][INFO][exchange-worker-#43][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=4, minorTo
      pVer=0], mvccCrd=MvccCoordinator [nodeId=d02069db-6d0b-4a40-b185-1d95fa330853, crdVer=1540479883228, topVer=AffinityTopo
      logyVersion [topVer=1, minorTopVer=0]], mvccCrdChange=false, crd=true, evt=NODE_JOINED, evtNode=d079c11d-bd7d-4d65-93ab-
      dee8d0b280e3, customEvt=null, allowMerge=true]
      [18:05:36,754][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], waitTime=0ms, futInfo=NA, mode=DISTRIBUTED]
      [18:05:36,754][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partitions release latch
      : ServerLatch [permits=0, pendingAcks=HashSet [], super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion [t
      opVer=4, minorTopVer=0]]]
      [18:05:36,754][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future
       [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], waitTime=0ms, futInfo=NA, mode=LOCAL]
      [18:05:36,754][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Logical recovery performed in 0 ms.
      [18:05:36,754][INFO][exchange-worker-#43][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=4, minorT
      opVer=0], crd=true]
      [18:05:37,423][INFO][grid-nio-worker-tcp-comm-1-#25][TcpCommunicationSpi] Accepted incoming communication connection [lo
      cAddr=/127.0.0.1:47100, rmtAddr=/127.0.0.1:61724]
      [18:05:37,439][INFO][sys-#62][GridDhtPartitionsExchangeFuture] Coordinator received single message [ver=AffinityTopology
      Version [topVer=4, minorTopVer=0], node=d079c11d-bd7d-4d65-93ab-dee8d0b280e3, allReceived=true]
      [18:05:37,439][SEVERE][sys-#62][GridDhtPartitionsExchangeFuture] Failed to notify listener: o.a.i.i.processors.cache.dis
      tributed.dht.preloader.GridDhtPartitionsExchangeFuture$2@3e21b701
      java.lang.NullPointerException
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.clientTopology(GridCachePartiti
      onExchangeManager.java:835)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.updateP
      artitionSingleMap(GridDhtPartitionsExchangeFuture.java:4031)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.process
      SingleMessage(GridDhtPartitionsExchangeFuture.java:2790)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$
      100(GridDhtPartitionsExchangeFuture.java:139)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply
      (GridDhtPartitionsExchangeFuture.java:2612)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply
      (GridDhtPartitionsExchangeFuture.java:2600)
              at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
              at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:355)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onRecei
      veSingleMessage(GridDhtPartitionsExchangeFuture.java:2600)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processSinglePartitionUpdate(Gr
      idCachePartitionExchangeManager.java:1656)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1200(GridCachePartitionE
      xchangeManager.java:142)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionE
      xchangeManager.java:375)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionE
      xchangeManager.java:333)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCacheP
      artitionExchangeManager.java:3102)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCacheP
      artitionExchangeManager.java:3081)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
              at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
              at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:119
      7)
              at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
              at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      [18:05:37,455][SEVERE][sys-#62][GridCacheIoManager] Failed processing message [senderId=d079c11d-bd7d-4d65-93ab-dee8d0b2
      80e3, msg=GridDhtPartitionsSingleMessage [parts=HashMap {-2100569601=GridDhtPartitionMap [moving=0, top=AffinityTopology
      Version [topVer=-1, minorTopVer=0], updateSeq=2, size=0], -1691533662=GridDhtPartitionMap [moving=0, top=AffinityTopolog
      yVersion [topVer=4, minorTopVer=0], updateSeq=2, size=0]}, partCntrs=HashMap {-2100569601=CachePartitionPartialCountersM
      ap {}, -1691533662=CachePartitionPartialCountersMap {}}, partsSizes=null, partHistCntrs=null, err=null, client=false, co
      mpress=false, finishMsg=null, activeQryTrackers=GridLongList [idx=0, arr=[]], super=GridDhtPartitionsAbstractMessage [ex
      chId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], discoEvt=null, nodeId=d079c11
      d, evt=NODE_JOINED], lastVer=GridCacheVersion [topVer=0, order=1540479929882, nodeOrder=0], super=GridCacheMessage [msgI
      d=1, depInfo=null, err=null, skipPrepare=false]]]]
      java.lang.NullPointerException
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.clientTopology(GridCachePartiti
      onExchangeManager.java:835)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.updateP
      artitionSingleMap(GridDhtPartitionsExchangeFuture.java:4031)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.process
      SingleMessage(GridDhtPartitionsExchangeFuture.java:2790)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$
      100(GridDhtPartitionsExchangeFuture.java:139)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply
      (GridDhtPartitionsExchangeFuture.java:2612)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply
      (GridDhtPartitionsExchangeFuture.java:2600)
              at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
              at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:355)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onRecei
      veSingleMessage(GridDhtPartitionsExchangeFuture.java:2600)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processSinglePartitionUpdate(Gr
      idCachePartitionExchangeManager.java:1656)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1200(GridCachePartitionE
      xchangeManager.java:142)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionE
      xchangeManager.java:375)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionE
      xchangeManager.java:333)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCacheP
      artitionExchangeManager.java:3102)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCacheP
      artitionExchangeManager.java:3081)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
              at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
              at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:119
      7)
              at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
              at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      [18:05:37,455][SEVERE][sys-#62][] Critical system error detected. Will be handled accordingly to configured handler [hnd
      =StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableS
      et [SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.NullPointerException]]
      java.lang.NullPointerException
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.clientTopology(GridCachePartiti
      onExchangeManager.java:835)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.updateP
      artitionSingleMap(GridDhtPartitionsExchangeFuture.java:4031)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.process
      SingleMessage(GridDhtPartitionsExchangeFuture.java:2790)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$
      100(GridDhtPartitionsExchangeFuture.java:139)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply
      (GridDhtPartitionsExchangeFuture.java:2612)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply
      (GridDhtPartitionsExchangeFuture.java:2600)
              at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
              at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:355)
              at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onRecei
      veSingleMessage(GridDhtPartitionsExchangeFuture.java:2600)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processSinglePartitionUpdate(Gr
      idCachePartitionExchangeManager.java:1656)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1200(GridCachePartitionE
      xchangeManager.java:142)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionE
      xchangeManager.java:375)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionE
      xchangeManager.java:333)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCacheP
      artitionExchangeManager.java:3102)
              at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCacheP
      artitionExchangeManager.java:3081)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
              at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
              at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
              at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:119
      7)
              at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
              at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      [18:05:37,455][WARNING][sys-#62][FailureProcessor] No deadlocked threads detected.
      [18:05:37,486][WARNING][sys-#62][FailureProcessor] Thread dump at 2018/10/25 18:05:37 MSK
      Thread [name="tcp-disco-sock-reader-#8", id=112, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at java.net.SocketInputStream.socketRead0(Native Method)
              at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
              at java.net.SocketInputStream.read(SocketInputStream.java:171)
              at java.net.SocketInputStream.read(SocketInputStream.java:141)
              at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
              at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
              at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
              - locked java.io.BufferedInputStream@69777ba6
              at o.a.i.marshaller.jdk.JdkMarshallerInputStreamWrapper.read(JdkMarshallerInputStreamWrapper.java:53)
              at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2663)
              at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2679)
              at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3156)
              at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862)
              at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358)
              at o.a.i.marshaller.jdk.JdkMarshallerObjectInputStream.<init>(JdkMarshallerObjectInputStream.java:43)
              at o.a.i.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:137)
              at o.a.i.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
              at o.a.i.i.util.IgniteUtils.unmarshal(IgniteUtils.java:10085)
              at o.a.i.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:6246)
              at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
      
      Thread [name="pool-3-thread-1", id=110, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@36a3688e, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="client-connector-#73", id=109, state=WAITING, blockCnt=0, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54dbe05e, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="pub-#72", id=108, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@33f5d231, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="pub-#70", id=106, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@33f5d231, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-#69", id=105, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@618d35a2, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-#68", id=104, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@618d35a2, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="vacuum-cleaner-#66", id=102, state=WAITING, blockCnt=0, waitCnt=3]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4cc7d605, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at o.a.i.i.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2184)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="vacuum-cleaner-#65", id=101, state=WAITING, blockCnt=2, waitCnt=10]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4cc7d605, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at o.a.i.i.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2184)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="vacuum-scheduler-#64", id=100, state=TIMED_WAITING, blockCnt=0, waitCnt=8]
              at java.lang.Thread.sleep(Native Method)
              at o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7774)
              at o.a.i.i.processors.cache.mvcc.MvccProcessorImpl$VacuumScheduler.body(MvccProcessorImpl.java:2158)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="query-#63", id=99, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@479516d3, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-#62", id=98, state=RUNNABLE, blockCnt=0, waitCnt=1]
              at sun.management.ThreadImpl.dumpThreads0(Native Method)
              at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:454)
              at o.a.i.i.util.IgniteUtils.dumpThreads(IgniteUtils.java:1364)
              at o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:132)
              - locked o.a.i.i.processors.failure.FailureProcessor@5a6c304a
              at o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:108)
              at o.a.i.i.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1071)
              at o.a.i.i.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
              at o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
              at o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
              at o.a.i.i.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
              at o.a.i.i.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
              at o.a.i.i.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
              at o.a.i.i.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
              at o.a.i.i.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
              at o.a.i.i.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
          Locked synchronizers:
              java.util.concurrent.ThreadPoolExecutor$Worker@69455d2e
      Thread [name="client-connector-#61", id=97, state=WAITING, blockCnt=0, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54dbe05e, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-#60", id=96, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@618d35a2, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-#59", id=95, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@618d35a2, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="jdbc-request-handler-worker-#58", id=94, state=WAITING, blockCnt=0, waitCnt=3]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@52c43e80, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at o.a.i.i.processors.odbc.jdbc.JdbcRequestHandlerWorker.body(JdbcRequestHandlerWorker.java:85)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="client-connector-#57", id=93, state=WAITING, blockCnt=0, waitCnt=1]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54dbe05e, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-#56", id=90, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@618d35a2, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="srvc-deploy-#55", id=87, state=WAITING, blockCnt=0, waitCnt=6]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@20c1ad23, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-#54", id=86, state=TIMED_WAITING, blockCnt=3, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@618d35a2, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="checkpoint-runner-#53", id=85, state=WAITING, blockCnt=4, waitCnt=781]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4f8d1466, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="checkpoint-runner-#52", id=84, state=WAITING, blockCnt=6, waitCnt=797]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4f8d1466, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="checkpoint-runner-#51", id=83, state=WAITING, blockCnt=3, waitCnt=823]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4f8d1466, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="checkpoint-runner-#50", id=82, state=WAITING, blockCnt=4, waitCnt=804]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4f8d1466, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="db-checkpoint-thread-#49", id=81, state=TIMED_WAITING, blockCnt=4, waitCnt=225]
          Lock [object=o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer@745883bd, ownerName=nu
      ll, ownerId=-1]
              at java.lang.Object.wait(Native Method)
              at o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.waitCheckpointEvent(GridCach
      eDatabaseSharedManager.java:3537)
              at o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseShared
      Manager.java:3085)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-#48", id=80, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@618d35a2, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="wal-segment-syncer-#47", id=79, state=TIMED_WAITING, blockCnt=0, waitCnt=108]
              at java.lang.Thread.sleep(Native Method)
              at o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7774)
              at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$WalSegmentSyncer.body(FileWriteAheadLogMana
      ger.java:3514)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="wal-file-archiver%null-#46", id=78, state=WAITING, blockCnt=0, waitCnt=1]
          Lock [object=o.a.i.i.processors.cache.persistence.wal.aware.SegmentCurrentStateStorage@79b9388, ownerName=null, owne
      rId=-1]
              at java.lang.Object.wait(Native Method)
              at java.lang.Object.wait(Object.java:502)
              at o.a.i.i.processors.cache.persistence.wal.aware.SegmentCurrentStateStorage.awaitSegment(SegmentCurrentStateSto
      rage.java:73)
              at o.a.i.i.processors.cache.persistence.wal.aware.SegmentCurrentStateStorage.waitNextSegmentForArchivation(Segme
      ntCurrentStateStorage.java:90)
              at o.a.i.i.processors.cache.persistence.wal.aware.SegmentAware.waitNextSegmentForArchivation(SegmentAware.java:7
      9)
              at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.body(FileWriteAheadLogManager.
      java:1719)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="rest-#45", id=77, state=TIMED_WAITING, blockCnt=1, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2a9e8eed, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="rest-#44", id=76, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2a9e8eed, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="session-timeout-worker-#34", id=51, state=TIMED_WAITING, blockCnt=0, waitCnt=54]
              at java.lang.Thread.sleep(Native Method)
              at o.a.i.i.processors.rest.GridRestProcessor$4.body(GridRestProcessor.java:495)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="exchange-worker-#43", id=74, state=TIMED_WAITING, blockCnt=2, waitCnt=35]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
              at o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:219)
              at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:160)
              at o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeMan
      ager.java:2695)
              at o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeMana
      ger.java:2539)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="ignite-update-notifier-timer", id=73, state=TIMED_WAITING, blockCnt=1, waitCnt=3]
          Lock [object=java.util.TaskQueue@d1791fc, ownerName=null, ownerId=-1]
              at java.lang.Object.wait(Native Method)
              at java.util.TimerThread.mainLoop(Timer.java:552)
              at java.util.TimerThread.run(Timer.java:505)
      
      Thread [name="upd-ver-checker", id=72, state=TIMED_WAITING, blockCnt=0, waitCnt=11]
              at java.lang.Thread.sleep(Native Method)
              at o.a.i.i.processors.cluster.GridUpdateNotifier$1.run(GridUpdateNotifier.java:115)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="disco-event-worker-#42", id=71, state=WAITING, blockCnt=0, waitCnt=38]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3d62be88, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:2856)
              at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:2825)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="tcp-disco-srvr-#3", id=69, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at java.net.DualStackPlainSocketImpl.accept0(Native Method)
              at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
              at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
              at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
              - locked java.net.SocksSocketImpl@66be6d7f
              at java.net.ServerSocket.implAccept(ServerSocket.java:545)
              at java.net.ServerSocket.accept(ServerSocket.java:513)
              at o.a.i.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:5845)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at o.a.i.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:5763)
              at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
      
      Thread [name="tcp-disco-msg-worker-#2", id=68, state=TIMED_WAITING, blockCnt=6, waitCnt=5916]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@38097ea3, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
              at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
              at o.a.i.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7183)
              at o.a.i.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at o.a.i.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
              at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
      
      Thread [name="disco-notifier-worker-#41", id=67, state=WAITING, blockCnt=2, waitCnt=44]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@920802f, ownerName=null, ownerId=
      -1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
              at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.jav
      a:2660)
              at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java
      :2705)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="Thread-9", id=66, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
              at sun.nio.ch.Iocp.access$300(Iocp.java:46)
              at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
              at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
          Locked synchronizers:
              java.util.concurrent.ThreadPoolExecutor$Worker@1849db1a
      Thread [name="Thread-8", id=65, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
              at sun.nio.ch.Iocp.access$300(Iocp.java:46)
              at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
              at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
          Locked synchronizers:
              java.util.concurrent.ThreadPoolExecutor$Worker@29a60c27
      Thread [name="Thread-7", id=64, state=RUNNABLE, blockCnt=1, waitCnt=0]
              at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
              at sun.nio.ch.Iocp.access$300(Iocp.java:46)
              at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
              at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
          Locked synchronizers:
              java.util.concurrent.ThreadPoolExecutor$Worker@79e18e38
      Thread [name="Thread-6", id=63, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
              at sun.nio.ch.Iocp.access$300(Iocp.java:46)
              at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
              at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
          Locked synchronizers:
              java.util.concurrent.ThreadPoolExecutor$Worker@3e2822
      Thread [name="Thread-5", id=62, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
              at sun.nio.ch.Iocp.access$300(Iocp.java:46)
              at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
              at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
          Locked synchronizers:
              java.util.concurrent.ThreadPoolExecutor$Worker@60fa3495
      Thread [name="Thread-4", id=61, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
              at sun.nio.ch.Iocp.access$300(Iocp.java:46)
              at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
              at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
          Locked synchronizers:
              java.util.concurrent.ThreadPoolExecutor$Worker@4602c2a9
      Thread [name="Thread-3", id=60, state=RUNNABLE, blockCnt=1, waitCnt=0]
              at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
              at sun.nio.ch.Iocp.access$300(Iocp.java:46)
              at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
              at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
          Locked synchronizers:
              java.util.concurrent.ThreadPoolExecutor$Worker@6a62689d
      Thread [name="Thread-2", id=59, state=RUNNABLE, blockCnt=1, waitCnt=0]
              at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
              at sun.nio.ch.Iocp.access$300(Iocp.java:46)
              at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
              at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
          Locked synchronizers:
              java.util.concurrent.ThreadPoolExecutor$Worker@4204541c
      Thread [name="Thread-1", id=58, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
              at sun.nio.ch.Iocp.access$300(Iocp.java:46)
              at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-data-loader-flusher-#40", id=57, state=WAITING, blockCnt=0, waitCnt=1]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@219ee97d, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2
      039)
              at java.util.concurrent.DelayQueue.take(DelayQueue.java:211)
              at o.a.i.i.processors.datastreamer.DataStreamProcessor$2.body(DataStreamProcessor.java:109)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-tcp-rest-3-#38", id=55, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@345eada4
              - locked java.util.Collections$UnmodifiableSet@475131ea
              - locked sun.nio.ch.WindowsSelectorImpl@4a7df0d7
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-tcp-rest-2-#37", id=54, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@1c7e3de4
              - locked java.util.Collections$UnmodifiableSet@5d8c8042
              - locked sun.nio.ch.WindowsSelectorImpl@131c2478
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-tcp-rest-1-#36", id=53, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@36c28584
              - locked java.util.Collections$UnmodifiableSet@5bbe1fa5
              - locked sun.nio.ch.WindowsSelectorImpl@62890a00
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-tcp-rest-0-#35", id=52, state=RUNNABLE, blockCnt=3, waitCnt=1]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@10c4cdcd
              - locked java.util.Collections$UnmodifiableSet@328894f4
              - locked sun.nio.ch.WindowsSelectorImpl@1960e637
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="nio-acceptor-tcp-rest-#39", id=56, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked sun.nio.ch.Util$3@33132611
              - locked java.util.Collections$UnmodifiableSet@50f2f1b
              - locked sun.nio.ch.WindowsSelectorImpl@7356247a
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929)
              at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-client-listener-3-#32", id=49, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@6d993579
              - locked java.util.Collections$UnmodifiableSet@4710b575
              - locked sun.nio.ch.WindowsSelectorImpl@3b9b2d7b
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-client-listener-2-#31", id=48, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@79acab86
              - locked java.util.Collections$UnmodifiableSet@5d3a6b37
              - locked sun.nio.ch.WindowsSelectorImpl@1fc3d6f8
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-client-listener-1-#30", id=47, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@7dd1c08e
              - locked java.util.Collections$UnmodifiableSet@613da429
              - locked sun.nio.ch.WindowsSelectorImpl@6f5018ea
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-client-listener-0-#29", id=46, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@6ea577ee
              - locked java.util.Collections$UnmodifiableSet@1d830f04
              - locked sun.nio.ch.WindowsSelectorImpl@1113fc71
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="nio-acceptor-client-listener-#33", id=50, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked sun.nio.ch.Util$3@7e6b31f1
              - locked java.util.Collections$UnmodifiableSet@3bb423d5
              - locked sun.nio.ch.WindowsSelectorImpl@3e4ae088
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929)
              at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="tcp-comm-worker-#1", id=45, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@30b47344, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4287)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at o.a.i.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237)
              at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
      
      Thread [name="grid-nio-worker-tcp-comm-3-#27", id=43, state=RUNNABLE, blockCnt=1, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@1777381a
              - locked java.util.Collections$UnmodifiableSet@dc70baa
              - locked sun.nio.ch.WindowsSelectorImpl@6460f43a
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-tcp-comm-2-#26", id=42, state=RUNNABLE, blockCnt=3, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@775d85a1
              - locked java.util.Collections$UnmodifiableSet@59c4d4b8
              - locked sun.nio.ch.WindowsSelectorImpl@6ceb1e71
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-tcp-comm-1-#25", id=41, state=RUNNABLE, blockCnt=1, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@f49e6f2
              - locked java.util.Collections$UnmodifiableSet@14a20a37
              - locked sun.nio.ch.WindowsSelectorImpl@6bbc7d0f
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-nio-worker-tcp-comm-0-#24", id=40, state=RUNNABLE, blockCnt=2, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked o.a.i.i.util.nio.SelectedSelectionKeySet@5f10fe9f
              - locked java.util.Collections$UnmodifiableSet@1faeedf9
              - locked sun.nio.ch.WindowsSelectorImpl@1b59af89
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
              at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="nio-acceptor-tcp-comm-#28", id=44, state=RUNNABLE, blockCnt=2, waitCnt=0]
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
              at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
              at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
              at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
              - locked sun.nio.ch.Util$3@1385fde8
              - locked java.util.Collections$UnmodifiableSet@4b390d01
              - locked sun.nio.ch.WindowsSelectorImpl@2390f0ba
              at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
              at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929)
              at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="grid-timeout-worker-#23", id=39, state=TIMED_WAITING, blockCnt=3, waitCnt=164]
          Lock [object=java.lang.Object@756326a6, ownerName=null, ownerId=-1]
              at java.lang.Object.wait(Native Method)
              at o.a.i.i.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:269)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="ignite-clock", id=38, state=TIMED_WAITING, blockCnt=0, waitCnt=5617]
              at java.lang.Thread.sleep(Native Method)
              at o.a.i.i.util.IgniteUtils$22.run(IgniteUtils.java:3379)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="pub-#19", id=34, state=TIMED_WAITING, blockCnt=1, waitCnt=2]
          Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@33f5d231, ownerName=null, ownerId
      =-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.j
      ava:2078)
              at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="jvm-pause-detector-worker", id=31, state=TIMED_WAITING, blockCnt=0, waitCnt=1360]
              at java.lang.Thread.sleep(Native Method)
              at o.a.i.i.LongJVMPauseDetector$1.run(LongJVMPauseDetector.java:100)
      
      Thread [name="data-streamer-stripe-7-#16", id=30, state=WAITING, blockCnt=0, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="data-streamer-stripe-6-#15", id=29, state=WAITING, blockCnt=1, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="data-streamer-stripe-5-#14", id=28, state=WAITING, blockCnt=0, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="data-streamer-stripe-4-#13", id=27, state=WAITING, blockCnt=0, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="data-streamer-stripe-3-#12", id=26, state=WAITING, blockCnt=3, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="data-streamer-stripe-2-#11", id=25, state=WAITING, blockCnt=3, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="data-streamer-stripe-1-#10", id=24, state=WAITING, blockCnt=0, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="data-streamer-stripe-0-#9", id=23, state=WAITING, blockCnt=3, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-stripe-7-#8", id=22, state=WAITING, blockCnt=0, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-stripe-6-#7", id=21, state=WAITING, blockCnt=0, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-stripe-5-#6", id=20, state=WAITING, blockCnt=0, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-stripe-4-#5", id=19, state=WAITING, blockCnt=3, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-stripe-3-#4", id=18, state=WAITING, blockCnt=0, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-stripe-2-#3", id=17, state=WAITING, blockCnt=3, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-stripe-1-#2", id=16, state=WAITING, blockCnt=0, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="sys-stripe-0-#1", id=15, state=WAITING, blockCnt=2, waitCnt=1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
              at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
              at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
              at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="RMI TCP Accept-0", id=14, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at java.net.DualStackPlainSocketImpl.accept0(Native Method)
              at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
              at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
              at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
              - locked java.net.SocksSocketImpl@49b5ff5f
              at java.net.ServerSocket.implAccept(ServerSocket.java:545)
              at java.net.ServerSocket.accept(ServerSocket.java:513)
              at sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:52)
              at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:405)
              at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:377)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="RMI TCP Accept-49148", id=13, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at java.net.DualStackPlainSocketImpl.accept0(Native Method)
              at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
              at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
              at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
              - locked java.net.SocksSocketImpl@4ae7b26d
              at java.net.ServerSocket.implAccept(ServerSocket.java:545)
              at java.net.ServerSocket.accept(ServerSocket.java:513)
              at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:405)
              at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:377)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="RMI TCP Accept-0", id=12, state=RUNNABLE, blockCnt=0, waitCnt=0]
              at java.net.DualStackPlainSocketImpl.accept0(Native Method)
              at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
              at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
              at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
              - locked java.net.SocksSocketImpl@6ed69eab
              at java.net.ServerSocket.implAccept(ServerSocket.java:545)
              at java.net.ServerSocket.accept(ServerSocket.java:513)
              at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:405)
              at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:377)
              at java.lang.Thread.run(Thread.java:748)
      
      Thread [name="Attach Listener", id=5, state=RUNNABLE, blockCnt=0, waitCnt=0]
      
      Thread [name="Signal Dispatcher", id=4, state=RUNNABLE, blockCnt=0, waitCnt=0]
      
      Thread [name="Finalizer", id=3, state=WAITING, blockCnt=193, waitCnt=5]
          Lock [object=java.lang.ref.ReferenceQueue$Lock@2ecc3c1, ownerName=null, ownerId=-1]
              at java.lang.Object.wait(Native Method)
              at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
              at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
              at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)
      
      Thread [name="Reference Handler", id=2, state=WAITING, blockCnt=6, waitCnt=4]
          Lock [object=java.lang.ref.Reference$Lock@64c59bc4, ownerName=null, ownerId=-1]
              at java.lang.Object.wait(Native Method)
              at java.lang.Object.wait(Object.java:502)
              at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
              at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
      
      Thread [name="main", id=1, state=WAITING, blockCnt=5, waitCnt=5]
          Lock [object=java.util.concurrent.CountDownLatch$Sync@134629eb, ownerName=null, ownerId=-1]
              at sun.misc.Unsafe.park(Native Method)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:8
      36)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer
      .java:997)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.j
      ava:1304)
              at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
              at o.a.i.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:334)
      
      
      
      [18:05:37,502][SEVERE][sys-#62][] JVM will be halted immediately due to the failure: [failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.NullPointerException]]
      
      

        Attachments

        1. PersistenceNodeRestartAfterCacheDropSelfTest.java
          3 kB
          Vladimir Ozerov
        2. ignite-gridparitition-nullpointer.zip
          357 kB
          Mahesh Renduchintala

          Activity

            People

            • Assignee:
              agoncharuk Alexey Goncharuk
              Reporter:
              skozlov Sergey Kozlov
            • Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: