Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-10052

Restart node during TX causes vacuum error.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.7
    • Fix Version/s: 2.7
    • Component/s: mvcc
    • Labels:
      None

      Description

      1. Start 2 nodes with PDS, activate , start sqlline
      2. Create table

      0: jdbc:ignite:thin://127.0.0.1/> create table t1 (a int, b int, primary key(a)) with "atomicity=TRANSACTIONAL_SNAPSHOT,
      backups=1";
      No rows affected (0,294 seconds)
      

      3. Open TX:

      0: jdbc:ignite:thin://127.0.0.1/> begin;
      No rows affected (0,007 seconds)
      0: jdbc:ignite:thin://127.0.0.1/> insert into t1 values (1,1);
      1 row affected (0,112 seconds)
      

      4. Stop and then start 2nd node
      5. Rollback TX and check the table data (no rows added):

      0: jdbc:ignite:thin://127.0.0.1/> rollback;
      No rows affected (0,011 seconds)
      0: jdbc:ignite:thin://127.0.0.1/> select * from t1;
      +--------------------------------+--------------------------------+
      |               A                |               B                |
      +--------------------------------+--------------------------------+
      +--------------------------------+--------------------------------+
      No rows selected (0,067 seconds)
      

      6. Start second TX that throws the exception:

      0: jdbc:ignite:thin://127.0.0.1/> begin;
      No rows affected (0,001 seconds)
      0: jdbc:ignite:thin://127.0.0.1/> insert into t1 values (1,1);
      Error: Failed to run update. Runtime failure on bounds: [lower=MvccMaxSearchRow [], upper=MvccMinSearchRow []] (state=50
      000,code=1)
      java.sql.SQLException: Failed to run update. Runtime failure on bounds: [lower=MvccMaxSearchRow [], upper=MvccMinSearchR
      ow []]
              at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:750)
              at org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:212)
              at org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:475)
              at sqlline.Commands.execute(Commands.java:823)
              at sqlline.Commands.sql(Commands.java:733)
              at sqlline.SqlLine.dispatch(SqlLine.java:795)
              at sqlline.SqlLine.begin(SqlLine.java:668)
              at sqlline.SqlLine.start(SqlLine.java:373)
              at sqlline.SqlLine.main(SqlLine.java:265)
      0: jdbc:ignite:thin://127.0.0.1/>
      

      The server nodes print out:

      [18:46:32,136][SEVERE][jdbc-request-handler-worker-#62][DmlStatementsProcessor] Error during update [localNodeId=319a2fda-1315-4bdf-8647-7ddee2d3342e]
      class org.apache.ignite.IgniteCheckedException: Runtime failure on bounds: [lower=MvccMaxSearchRow [], upper=MvccMinSearchRow []]
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1061)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:1919)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.mvccUpdate(GridCacheOffheapManager.java:1840)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:530)
      	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.mvccSet(GridCacheMapEntry.java:1104)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.continueLoop(GridDhtTxAbstractEnlistFuture.java:460)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.init(GridDhtTxAbstractEnlistFuture.java:368)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearTxQueryResultsEnlistRequest(GridDhtTransactionalCacheAdapter.java:2000)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.access$700(GridDhtTransactionalCacheAdapter.java:112)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$12.apply(GridDhtTransactionalCacheAdapter.java:215)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$12.apply(GridDhtTransactionalCacheAdapter.java:213)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
      	at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
      	at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: Unexpected state: [state=0, rowVer=1540827914710:6:1, txVer=1540827914710:21:1, localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079]
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:297)
      	at org.apache.ignite.internal.processors.cache.tree.mvcc.data.MvccUpdateDataRow.visit(MvccUpdateDataRow.java:413)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3021)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.init(BPlusTree.java:2990)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.notFound(BPlusTree.java:2946)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:306)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5615)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:271)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5600)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:159)
      	at org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:334)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visitDown(BPlusTree.java:2585)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doVisit(BPlusTree.java:2552)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$8200(BPlusTree.java:90)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3148)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.access$5100(BPlusTree.java:2908)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1058)
      	... 23 more
      [18:46:32,138][SEVERE][jdbc-request-handler-worker-#62][JdbcRequestHandler] Failed to execute SQL query [reqId=0, req=JdbcQueryExecuteRequest [schemaName=PUBLIC, pageSize=1024, maxRows=0, sqlQry=insert into t1 values (1,1), args=Object[] [], stmtType=ANY_STATEMENT_TYPE, autoCommit=true]]
      class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to run update. Runtime failure on bounds: [lower=MvccMaxSearchRow [], upper=MvccMinSearchRow []]
      	at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.executeUpdateStatement(DmlStatementsProcessor.java:593)
      	at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFields(DmlStatementsProcessor.java:185)
      	at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFieldsDistributed(DmlStatementsProcessor.java:358)
      	at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.doRunPrepared(IgniteH2Indexing.java:2239)
      	at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:2158)
      	at org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2141)
      	at org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2136)
      	at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
      	at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2713)
      	at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2150)
      	at org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.executeQuery(JdbcRequestHandler.java:511)
      	at org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.doHandle(JdbcRequestHandler.java:245)
      	at org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandlerWorker.body(JdbcRequestHandlerWorker.java:90)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: class org.apache.ignite.IgniteCheckedException: Runtime failure on bounds: [lower=MvccMaxSearchRow [], upper=MvccMinSearchRow []]
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1061)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:1919)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.mvccUpdate(GridCacheOffheapManager.java:1840)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:530)
      	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.mvccSet(GridCacheMapEntry.java:1104)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.continueLoop(GridDhtTxAbstractEnlistFuture.java:460)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.init(GridDhtTxAbstractEnlistFuture.java:368)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearTxQueryResultsEnlistRequest(GridDhtTransactionalCacheAdapter.java:2000)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.access$700(GridDhtTransactionalCacheAdapter.java:112)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$12.apply(GridDhtTransactionalCacheAdapter.java:215)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$12.apply(GridDhtTransactionalCacheAdapter.java:213)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
      	at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
      	at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505)
      	... 2 more
      Caused by: class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: Unexpected state: [state=0, rowVer=1540827914710:6:1, txVer=1540827914710:21:1, localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079]
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:297)
      	at org.apache.ignite.internal.processors.cache.tree.mvcc.data.MvccUpdateDataRow.visit(MvccUpdateDataRow.java:413)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3021)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.init(BPlusTree.java:2990)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.notFound(BPlusTree.java:2946)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:306)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5615)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:271)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5600)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:159)
      	at org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:334)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visitDown(BPlusTree.java:2585)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doVisit(BPlusTree.java:2552)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$8200(BPlusTree.java:90)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3148)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.access$5100(BPlusTree.java:2908)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1058)
      	... 23 more
      
      [18:46:20,439][SEVERE][vacuum-scheduler-#47][MvccProcessorImpl] Vacuum error.
      class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: Unexpected state: [state=0, rowVer=1540827914710:6:536870913, localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079]
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:281)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3206)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3180)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:279)
      	at org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:256)
      	at org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.updateDataRow(AbstractFreeList.java:555)
      	at org.apache.ignite.internal.processors.cache.persistence.RowStore.updateDataRow(RowStore.java:143)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxState(IgniteCacheOffheapManagerImpl.java:2449)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.updateTxState(GridCacheOffheapManager.java:1917)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImpl.java:2415)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccProcessorImpl.java:2300)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2198)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
      	at java.lang.Thread.run(Thread.java:748)
      [18:46:20,582][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, checkpointLockHoldTime=12ms, reason='timeout']
      [18:46:20,988][INFO][exchange-worker-#43][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], mvccCrd=MvccCoordinator [nodeId=319a2fda-1315-4bdf-8647-7ddee2d3342e, crdVer=1540827914710, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0]], mvccCrdChange=false, crd=false, evt=DISCOVERY_CUSTOM_EVT, evtNode=319a2fda-1315-4bdf-8647-7ddee2d3342e, customEvt=CacheAffinityChangeMessage [id=f4aff70c661-30eb588f-bc2f-4094-86c4-e7070c6fcb1e, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], exchId=null, partsMsg=null, exchangeNeeded=true], allowMerge=false]
      [18:46:20,998][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=DISTRIBUTED]
      [18:46:21,004][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partitions release latch: ClientLatch [coordinator=TcpDiscoveryNode [id=319a2fda-1315-4bdf-8647-7ddee2d3342e, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540827969985, loc=false, ver=2.7.1#20181029-sha1:83c27088, isClient=false], ackSent=true, super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1]]]
      [18:46:21,004][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=LOCAL]
      [18:46:21,005][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] Logical recovery performed in 0 ms.
      [18:46:21,015][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] Sending Single Message performed in 3 ms.
      [18:46:21,016][INFO][exchange-worker-#43][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], crd=false]
      [18:46:21,035][INFO][sys-#64][GridDhtPartitionsExchangeFuture] Received full message, will finish exchange [node=319a2fda-1315-4bdf-8647-7ddee2d3342e, resVer=AffinityTopologyVersion [topVer=4, minorTopVer=1]]
      [18:46:21,039][INFO][sys-#64][GridDhtPartitionsExchangeFuture] Affinity changes applied in 0 ms.
      [18:46:21,044][INFO][sys-#64][GridDhtPartitionsExchangeFuture] Full map updating for 2 groups performed in 2 ms.
      [18:46:21,044][INFO][sys-#64][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], err=null]
      [18:46:21,050][INFO][sys-#64][GridDhtPartitionsExchangeFuture] Completed partition exchange [localNode=4d66a9d5-4352-495c-9c64-405ccdf8a079, exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode [id=319a2fda-1315-4bdf-8647-7ddee2d3342e, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540827969985, loc=false, ver=2.7.1#20181029-sha1:83c27088, isClient=false], done=true], topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], durationFromInit=66]
      [18:46:21,052][INFO][exchange-worker-#43][GridCachePartitionExchangeManager] Skipping rebalancing (no affinity changes) [top=AffinityTopologyVersion [topVer=4, minorTopVer=1], rebTopVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], evt=DISCOVERY_CUSTOM_EVT, evtNode=319a2fda-1315-4bdf-8647-7ddee2d3342e, client=false]
      [18:46:21,586][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, checkpointLockHoldTime=10ms, reason='timeout']
      [18:46:22,595][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, checkpointLockHoldTime=9ms, reason='timeout']
      [18:46:24,602][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, checkpointLockHoldTime=7ms, reason='timeout']
      [18:46:25,406][SEVERE][vacuum-scheduler-#47][MvccProcessorImpl] Vacuum error.
      class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: Unexpected state: [state=0, rowVer=1540827914710:6:536870913, localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079]
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:281)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3206)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3180)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:279)
      	at org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:256)
      	at org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.updateDataRow(AbstractFreeList.java:555)
      	at org.apache.ignite.internal.processors.cache.persistence.RowStore.updateDataRow(RowStore.java:143)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxState(IgniteCacheOffheapManagerImpl.java:2449)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.updateTxState(GridCacheOffheapManager.java:1917)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImpl.java:2415)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccProcessorImpl.java:2300)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2198)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
      	at java.lang.Thread.run(Thread.java:748)
      [18:46:26,612][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, checkpointLockHoldTime=5ms, reason='timeout']
      [18:46:29,632][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, checkpointLockHoldTime=6ms, reason='timeout']
      [18:46:30,406][SEVERE][vacuum-scheduler-#47][MvccProcessorImpl] Vacuum error.
      class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: Unexpected state: [state=0, rowVer=1540827914710:6:536870913, localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079]
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:281)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3206)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3180)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:279)
      	at org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:256)
      	at org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.updateDataRow(AbstractFreeList.java:555)
      	at org.apache.ignite.internal.processors.cache.persistence.RowStore.updateDataRow(RowStore.java:143)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxState(IgniteCacheOffheapManagerImpl.java:2449)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.updateTxState(GridCacheOffheapManager.java:1917)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImpl.java:2415)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccProcessorImpl.java:2300)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2198)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
      	at java.lang.Thread.run(Thread.java:748)
      [18:46:35,406][SEVERE][vacuum-scheduler-#47][MvccProcessorImpl] Vacuum error.
      class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: Unexpected state: [state=0, rowVer=1540827914710:6:536870913, localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079]
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:281)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3206)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3180)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:279)
      	at org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:256)
      	at org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.updateDataRow(AbstractFreeList.java:555)
      	at org.apache.ignite.internal.processors.cache.persistence.RowStore.updateDataRow(RowStore.java:143)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxState(IgniteCacheOffheapManagerImpl.java:2449)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.updateTxState(GridCacheOffheapManager.java:1917)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImpl.java:2415)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccProcessorImpl.java:2300)
      	at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2198)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
      	at java.lang.Thread.run(Thread.java:748)
      [18:46:38,674][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, checkpointLockHoldTime=4ms, reason='timeout']
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                amashenkov Andrey Mashenkov
                Reporter:
                skozlov Sergey Kozlov
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: