Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-11953

BTree corruption caused by byte array values

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8
    • Component/s: None
    • Labels:
      None

      Description

      In some cases for caches with cache group, we can get BTree corruption exception.

      09:53:58,890][SEVERE][sys-stripe-10-#11][] Critical system error detected. Will be handled accordingly to configured handler [hnd=CustomFailureHandler [ignoreCriticalErrors=false, disabled=false][StopNodeOrHaltFailureHandler [tryStop=false, timeout=0]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.transactions.IgniteTxHeuristicCheckedException: Committing a transaction has produced runtime exception]]class org.apache.ignite.internal.transactions.IgniteTxHeuristicCheckedException: Committing a transaction has produced runtime exception
      	at org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.heuristicException(IgniteTxAdapter.java:800)
      	at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:922)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:799)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.localFinish(GridDhtTxLocal.java:608)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.finishTx(GridDhtTxLocal.java:478)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.commitDhtLocalAsync(GridDhtTxLocal.java:535)
      	at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:1055)
      	at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:931)
      	at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:887)
      	at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$200(IgniteTxHandler.java:117)
      	at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:209)
      	at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:207)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1129)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:594)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:393)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:319)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1568)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1196)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
      	at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1092)
      	at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:504)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: Runtime failure on search row: SearchRow [key=KeyCacheObjectImpl [part=427, val=Grkg1DUF3yQE6tC9Se50mi5w.T, hasValBytes=true], hash=1872857770, cacheId=-420893003]
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1811)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1620)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1603)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:2131)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:442)
      	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:4263)
      	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:4241)
      	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1541)
      	at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:721)
      	... 23 more
      Caused by: class org.apache.ignite.binary.BinaryObjectException: Not enough data to read the value [position=0, requiredBytes=1, remainingBytes=0]
      	at org.apache.ignite.internal.binary.streams.BinaryAbstractInputStream.ensureEnoughData(BinaryAbstractInputStream.java:304)
      	at org.apache.ignite.internal.binary.streams.BinaryAbstractInputStream.readByte(BinaryAbstractInputStream.java:34)
      	at org.apache.ignite.internal.binary.BinaryUtils.unmarshal(BinaryUtils.java:1830)
      	at org.apache.ignite.internal.binary.BinaryUtils.unmarshal(BinaryUtils.java:1819)
      	at org.apache.ignite.internal.binary.BinaryUtils.unmarshal(BinaryUtils.java:1810)
      	at org.apache.ignite.internal.binary.GridBinaryMarshaller.unmarshal(GridBinaryMarshaller.java:271)
      	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.unmarshal(CacheObjectBinaryProcessorImpl.java:928)
      	at org.apache.ignite.internal.processors.cacheobject.IgniteCacheObjectProcessorImpl.toKeyCacheObject(IgniteCacheObjectProcessorImpl.java:162)
      	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toKeyCacheObject(CacheObjectBinaryProcessorImpl.java:995)
      	at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFullRow(CacheDataRowAdapter.java:334)
      	at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:166)
      	at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:107)
      	at org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:54)
      	at org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.keySearchRow(CacheDataRowStore.java:57)
      	at org.apache.ignite.internal.processors.cache.tree.AbstractDataLeafIO.getLookupRow(AbstractDataLeafIO.java:164)
      	at org.apache.ignite.internal.processors.cache.tree.AbstractDataLeafIO.getLookupRow(AbstractDataLeafIO.java:34)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Put.insertWithSplit(BPlusTree.java:3500)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Put.insert(BPlusTree.java:3424)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Put.access$2100(BPlusTree.java:3298)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Insert.run0(BPlusTree.java:440)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Insert.run0(BPlusTree.java:421)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5650)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5635)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:343)
      	at org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:275)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$12000(BPlusTree.java:89)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Put.tryInsert(BPlusTree.java:3618)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Put.access$7300(BPlusTree.java:3298)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.onNotFound(BPlusTree.java:3856)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$6000(BPlusTree.java:3648)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1896)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1871)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1778)
      	... 31 more
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                DmitriyGovorukhin Dmitriy Govorukhin
                Reporter:
                DmitriyGovorukhin Dmitriy Govorukhin
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: