Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-12593

Corruption of B+Tree caused by byte array values and TTL

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8
    • Component/s: None
    • Labels:
      None
    • Ignite Flags:
      Docs Required, Release Notes Required

      Description

      It seems that the following set of parameters may lead to a corruption of B+Tree:

      • persistence is enabled
      • TTL is enabled
      • Expiry policy - AccessedExpiryPolicy 1 sec.
      • cache value type is byte[]
      • all caches belong to the same cache group

      Example of the stack trace:

      [2019-07-16 21:13:19,288][ERROR][sys-stripe-2-#46%db.IgnitePdsWithTtlDeactivateOnHighloadTest1%][IgniteTestResources] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple [val1=-1237460590, val2=281586645860358]], msg=Runtime failure on search row: SearchRow [key=KeyCacheObjectImpl [part=26, val=378, hasValBytes=true], hash=378, cacheId=-1806498247]]]]
      class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple [val1=-1237460590, val2=281586645860358]], msg=Runtime failure on search row: SearchRow [key=KeyCacheObjectImpl [part=26, val=378, hasValBytes=true], hash=378, cacheId=-1806498247]]
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:5910)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1859)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1662)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1645)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:2410)
      	at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:445)
      	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2309)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2570)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2030)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1848)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1668)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateRequest(GridDhtAtomicCache.java:3235)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$400(GridDhtAtomicCache.java:139)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$5.apply(GridDhtAtomicCache.java:273)
      	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$5.apply(GridDhtAtomicCache.java:268)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1141)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
      	at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1558)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1186)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
      	at org.apache.ignite.internal.managers.communication.GridIoManager$8.run(GridIoManager.java:1083)
      	at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:559)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.AssertionError: Invalid entry [entry=AtomicCacheUpdateClosure [entry=GridDhtCacheEntry [rdrs=ReaderId[] [], part=26, super=GridDistributedCacheEntry [super=GridCacheMapEntry [key=KeyCacheObjectImpl [part=26, val=378, hasValBytes=true], val=CacheObjectByteArrayImpl [arrLen=1024], ver=GridCacheVersion [topVer=174780794, order=1563300795497, nodeOrder=2], hash=378, extras=GridCacheTtlEntryExtras [ttl=0, expireTime=1563300799721], flags=3]]], topVer=AffinityTopologyVersion [topVer=2, minorTopVer=1], newVer=GridCacheVersion [topVer=174780794, order=1563300801949, nodeOrder=2], op=UPDATE, readThrough=false, writeThrough=false, keepBinary=false, expiryPlc=CacheExpiryPolicy [], primary=true, verCheck=false, explicitTtl=-1, explicitExpireTime=-1, conflictVer=null, conflictResolve=true, intercept=false, updateCntr=null, skipInterceptorOnConflict=false, updateRes=null, treeOp=PUT, newRow=null, oldRow=CacheDataRowAdapter [key=KeyCacheObjectImpl [part=26, val=378, hasValBytes=true], val=CacheObjectByteArrayImpl [arrLen=1024], expireTime=1563300799721, ver=GridCacheVersion [topVer=174780794, order=1563300795497, nodeOrder=2], cacheId=-1806498247, link=0001001a0000002a], oldRowExpiredFlag=false, wasIntercepted=false], locNodeId=c0cf7462-d27b-4a46-b157-fb6c02d00001]
      	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:6378)
      	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:6137)
      	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:5828)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:3800)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$5700(BPlusTree.java:3694)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1948)
      	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1826)
      	... 26 more
      

      The reproducer is attached.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                akalashnikov Anton Kalashnikov
                Reporter:
                akalashnikov Anton Kalashnikov
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m