[IGNITE-10571] Crash in exchange worker during memory recovery on rebalance

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Affects Version/s: 2.4
• Assignee: Alexey Goncharuk
• Reporter: Max Shonichev
• Created: 06/Dec/18 10:14
• Updated: 26/Dec/18 10:16
• Resolved: 26/Dec/18 10:16

Description

1. start grid (2 nodes), preload data, start 1 client node
2. kill 1 server node
3. wait 10 sec
4. restart server node

upon 2-3 iteration, node crashes JVM during memory recovery

Stack: [0x00007fdcbb2f5000,0x00007fdcbb3f6000],  sp=0x00007fdcbb3f4160,  free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
v  ~StubRoutines::jshort_disjoint_arraycopy
J 3894 C2 org.apache.ignite.internal.processors.cache.persistence.tree.io.BPlusLeafIO.copyItems(JJIIIZ)V (55 bytes) @ 0x00007fdeadba2949 [0x00007fdeadba28e0+0x69]
J 3893 C2 org.apache.ignite.internal.processors.cache.persistence.tree.io.BPlusIO.insert(JILjava/lang/Object;[BJZ)[B (44 bytes) @ 0x00007fdeadba2514 [0x00007fdeadba24a0+0x74]
J 3887 C2 org.apache.ignite.internal.pagemem.wal.record.delta.InsertRecord.applyDelta(Lorg/apache/ignite/internal/pagemem/PageMemory;J)V (24 bytes) @ 0x00007fdeadbab0c8 [0x00007fdeadbab060+0x68]
J 3895% C2 org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(Lorg/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager$CheckpointStatus;ZLorg/apache/ignite/internal/processors/cache/persistence/pagemem/PageMemoryEx;)Lorg/apache/ignite/internal/pagemem/wal/WALPointer; (1298 bytes) @ 0x00007fdeadb3eb00 [0x00007fdeadb3d3a0+0x1760]
j  org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(Lorg/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager$CheckpointStatus;)Lorg/apache/ignite/internal/pagemem/wal/WALPointer;+13
j  org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(Ljava/util/List;)V+173
j  org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin()V+147
j  org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(Z)V+794
j  org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body()V+547
j  org.apache.ignite.internal.util.worker.GridWorker.run()V+82
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub
V  [libjvm.so+0x697a76]
[error occurred during error reporting (printing native stack), id 0xb]

Issue Links

• is duplicated by: IGNITE-9303 PageSnapshot can contain wrong pageId tag when not dirty page is recycling