Details
Description
2019-09-05 17:54:46.576 INFO STDERR [Thread-18] : <GridCompoundFuture> [17:54:46] (err) Failed to execute compound future reducer: GridCompoundFuture [rdc=null, initFlag=0, lsnrCalls=0, done=false, cancelled=false, err=null, futs=[false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, true]]class org.apache.ignite.IgniteCheckedException: class org.apache.ignite.transactions.TransactionRollbackException: Transaction has been rolled back: 108bb6a2d61-00000000-0aad-7c5d-0000-000000000001
2019-09-05 17:54:46.576 INFO STDERR [Thread-18] : <GridCompoundFuture> at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7429)
2019-09-05 17:54:46.576 INFO STDERR [Thread-18] : <GridCompoundFuture> at org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:975)
2019-09-05 17:54:46.576 INFO STDERR [Thread-18] : <GridCompoundFuture> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : <GridCompoundFuture> at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505)
2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : <GridCompoundFuture> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : <GridCompoundFuture> at java.lang.Thread.run(Thread.java:745)
2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : <Throwable> Caused by: javax.cache.CacheException: class org.apache.ignite.transactions.TransactionRollbackException: Transaction has been rolled back: 108bb6a2d61-00000000-0aad-7c5d-0000-000000000001
2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1337)
2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1758)
2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1108)
2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:820)
2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.datastreamer.DataStreamerCacheUpdaters$Individual.receive(DataStreamerCacheUpdaters.java:121)
2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140)
2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6817)
2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967)
2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : <Throwable> ... 4 more
2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : <Throwable> Caused by: class org.apache.ignite.transactions.TransactionRollbackException: Transaction has been rolled back: 108bb6a2d61-00000000-0aad-7c5d-0000-000000000001
2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.util.IgniteUtils$11.apply(IgniteUtils.java:920)
2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.util.IgniteUtils$11.apply(IgniteUtils.java:918)
2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : <Throwable> ... 12 more
2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : <Throwable> Caused by: class org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException: Transaction has been rolled back: 108bb6a2d61-00000000-0aad-7c5d-0000-000000000001
2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4248)
2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2468)
2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2449)
2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2426)
2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1105)
2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : <Throwable> ... 9 more
2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : <Throwable> Caused by: class org.apache.ignite.IgniteCheckedException: Runtime failure on bounds: [lower=MvccMaxSearchRow [], upper=MvccMinSearchRow []]
2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1061)
2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:1939)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:532)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.mvccSet(GridCacheMapEntry.java:1104)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.continueLoop(GridDhtTxAbstractEnlistFuture.java:456)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.init(GridDhtTxAbstractEnlistFuture.java:364)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.enlistLocal(GridNearTxEnlistFuture.java:518)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.sendBatch(GridNearTxEnlistFuture.java:413)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.sendNextBatches(GridNearTxEnlistFuture.java:168)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.map(GridNearTxEnlistFuture.java:144)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxAbstractEnlistFuture.mapOnTopology(GridNearTxAbstractEnlistFuture.java:331)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxAbstractEnlistFuture.init(GridNearTxAbstractEnlistFuture.java:246)
2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.updateAsync(GridNearTxLocal.java:2099)
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.mvccPutAllAsync0(GridNearTxLocal.java:802)
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAsync0(GridNearTxLocal.java:582)
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAsync(GridNearTxLocal.java:448)
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2470)
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2468)
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4233)
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> ... 13 more
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> Caused by: class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: Unexpected state: [state=0, rowVer=1567661724689:237333229:1, txVer=1567661724689:237689217:1, localNodeId=eb509bae-864c-46bc-a85e-57928b639f67]
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:317)
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:304)
2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.tree.mvcc.data.MvccUpdateDataRow.visit(MvccUpdateDataRow.java:413)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3021)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.init(BPlusTree.java:2990)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.notFound(BPlusTree.java:2946)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:306)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5615)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:271)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5600)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:159)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:334)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visitDown(BPlusTree.java:2585)
2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visitDown(BPlusTree.java:2594)
2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doVisit(BPlusTree.java:2552)
2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$8200(BPlusTree.java:90)
2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3148)
2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.access$5100(BPlusTree.java:2908)
2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : <Throwable> at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1058)
2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : <Throwable> ... 31 more
2019-09-05 17:54:46.692 ERROR org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl vacuum-scheduler-#45%rates-marple-pnlswaps% : Vacuum error.
class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: Unexpected state: [state=0, rowVer=1567661724689:237333229:1, localNodeId=eb509bae-864c-46bc-a85e-57928b639f67]
at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:317)
at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:288)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3312)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3286)
at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:279)
at org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:256)
at org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.updateDataRow(AbstractFreeList.java:555)
at org.apache.ignite.internal.processors.cache.persistence.RowStore.updateDataRow(RowStore.java:138)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxState(IgniteCacheOffheapManagerImpl.java:2469)
at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImpl.java:2425)
at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccProcessorImpl.java:2285)
at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2208)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:745)
2019-09-05 17:54:47.652 ERROR root data-streamer-stripe-2-#11%rates-marple-pnlswaps% : Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.IgniteCheckedException: Maximum number of retries 1000 reached for Remove operation (the tree may be corrupted). Increase IGNITE_BPLUS_TREE_LOCK_RETRIES system property if you regularly see this message (current value is 1000).]]
class org.apache.ignite.IgniteCheckedException: Maximum number of retries 1000 reached for Remove operation (the tree may be corrupted). Increase IGNITE_BPLUS_TREE_LOCK_RETRIES system property if you regularly see this message (current value is 1000).
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Get.checkLockRetry(BPlusTree.java:2813)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.removeDown(BPlusTree.java:1997)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.removeDown(BPlusTree.java:2023)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.removeDown(BPlusTree.java:2023)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doRemove(BPlusTree.java:1934)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.removex(BPlusTree.java:1764)
at org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.removex(H2TreeIndex.java:345)
at org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.remove(GridH2Table.java:521)
at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.remove(IgniteH2Indexing.java:797)
at org.apache.ignite.internal.processors.query.GridQueryProcessor.remove(GridQueryProcessor.java:2586)
at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.remove(GridCacheQueryManager.java:435)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cleanup(IgniteCacheOffheapManagerImpl.java:2445)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:2034)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:532)
at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.mvccSet(GridCacheMapEntry.java:1104)
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.continueLoop(GridDhtTxAbstractEnlistFuture.java:456)
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.init(GridDhtTxAbstractEnlistFuture.java:364)
at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.enlistLocal(GridNearTxEnlistFuture.java:518)
at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.sendBatch(GridNearTxEnlistFuture.java:413)
at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.sendNextBatches(GridNearTxEnlistFuture.java:168)
at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxEnlistFuture.map(GridNearTxEnlistFuture.java:144)
at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxAbstractEnlistFuture.mapOnTopology(GridNearTxAbstractEnlistFuture.java:331)
at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxAbstractEnlistFuture.init(GridNearTxAbstractEnlistFuture.java:246)
at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.updateAsync(GridNearTxLocal.java:2099)
at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.mvccPutAllAsync0(GridNearTxLocal.java:802)
at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAsync0(GridNearTxLocal.java:582)
at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAsync(GridNearTxLocal.java:448)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2470)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2468)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4233)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2468)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2449)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2426)
at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1105)
at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:820)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerCacheUpdaters$Individual.receive(DataStreamerCacheUpdaters.java:121)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140)
at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6817)
at org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:745)
Attachments
Issue Links
- is duplicated by
-
IGNITE-12144 Vacuum error. class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException
- Resolved