[IGNITE-9489] CorruptedTreeException on index create.

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 2.4, 2.5, 2.6
• Fix Version/s: 2.7.6
• Component/s: cache, sql
• Assignee: Ivan Pavlukhin
• Reporter: Igor Seliverstov
• Created: 06/Sep/18 17:00
• Updated: 17/Dec/19 12:20
• Resolved: 17/Dec/19 12:20

Description

Currently on dynamic index drop with enabled persistence H2TreeIndex instances aren't destroyed. That means that their root pages aren't removed from meta tree (see org.apache.ignite.internal.processors.cache.persistence.IndexStorageImpl#getOrAllocateForTree) and reused on subsequent dynamic index create that leads CorruptedTreeException on initial index rebuild because there are some items with broken links on the root page.

Reproducer attached.

Error log:

Error during parallel index create/rebuild.
org.h2.message.DbException: Внутренняя ошибка: "class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: Runtime failure on row: Row@7745722d[ key: org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$KeyClass [idHash=2038596277, hash=-1388553726, id=1], val: org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$ValueClass [idHash=2109544797, hash=-898815788, field1=val1], ver: GridCacheVersion [topVer=147733489, order=1536253488473, nodeOrder=2] ][ 1, val1, null ]"
General error: "class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: Runtime failure on row: Row@7745722d[ key: org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$KeyClass [idHash=2038596277, hash=-1388553726, id=1], val: org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$ValueClass [idHash=2109544797, hash=-898815788, field1=val1], ver: GridCacheVersion [topVer=147733489, order=1536253488473, nodeOrder=2] ][ 1, val1, null ]" [50000-195]
	at org.h2.message.DbException.get(DbException.java:168)
	at org.h2.message.DbException.convert(DbException.java:295)
	at org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.putx(H2TreeIndex.java:251)
	at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$3.apply(IgniteH2Indexing.java:890)
	at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.updateIndex(GridCacheMapEntry.java:4320)
	at org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.processKey(SchemaIndexCacheVisitorImpl.java:244)
	at org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.processPartition(SchemaIndexCacheVisitorImpl.java:207)
	at org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.processPartitions(SchemaIndexCacheVisitorImpl.java:166)
	at org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.access$100(SchemaIndexCacheVisitorImpl.java:50)
	at org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl$AsyncWorker.body(SchemaIndexCacheVisitorImpl.java:317)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.h2.jdbc.JdbcSQLException: Внутренняя ошибка: "class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: Runtime failure on row: Row@7745722d[ key: org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$KeyClass [idHash=2038596277, hash=-1388553726, id=1], val: org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$ValueClass [idHash=2109544797, hash=-898815788, field1=val1], ver: GridCacheVersion [topVer=147733489, order=1536253488473, nodeOrder=2] ][ 1, val1, null ]"
General error: "class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: Runtime failure on row: Row@7745722d[ key: org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$KeyClass [idHash=2038596277, hash=-1388553726, id=1], val: org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$ValueClass [idHash=2109544797, hash=-898815788, field1=val1], ver: GridCacheVersion [topVer=147733489, order=1536253488473, nodeOrder=2] ][ 1, val1, null ]" [50000-195]
	at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
	... 12 more
Caused by: class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: Runtime failure on row: Row@7745722d[ key: org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$KeyClass [idHash=2038596277, hash=-1388553726, id=1], val: org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$ValueClass [idHash=2109544797, hash=-898815788, field1=val1], ver: GridCacheVersion [topVer=147733489, order=1536253488473, nodeOrder=2] ][ 1, val1, null ]
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doPut(BPlusTree.java:2285)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putx(BPlusTree.java:2232)
	at org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.putx(H2TreeIndex.java:248)
	... 9 more
Caused by: java.lang.AssertionError: itemId=2, directCnt=1, indirectCnt=0, page=0001033e00000007 [3991][][free=3935]
	at org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO.getDataOffset(AbstractDataPageIO.java:449)
	at org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO.readPayload(AbstractDataPageIO.java:493)
	at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:156)
	at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:108)
	at org.apache.ignite.internal.processors.query.h2.database.H2RowFactory.getRow(H2RowFactory.java:62)
	at org.apache.ignite.internal.processors.query.h2.database.H2Tree.createRowFromLink(H2Tree.java:159)
	at org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO.getLookupRow(AbstractH2ExtrasLeafIO.java:154)
	at org.apache.ignite.internal.processors.query.h2.database.io.AbstractH2ExtrasLeafIO.getLookupRow(AbstractH2ExtrasLeafIO.java:36)
	at org.apache.ignite.internal.processors.query.h2.database.H2Tree.getRow(H2Tree.java:190)
	at org.apache.ignite.internal.processors.query.h2.database.H2Tree.getRow(H2Tree.java:48)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.getRow(BPlusTree.java:5118)
	at org.apache.ignite.internal.processors.query.h2.database.H2Tree.compare(H2Tree.java:267)
	at org.apache.ignite.internal.processors.query.h2.database.H2Tree.compare(H2Tree.java:48)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.compare(BPlusTree.java:5105)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findInsertionPoint(BPlusTree.java:5025)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$1300(BPlusTree.java:90)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:291)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5615)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:271)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5600)
	at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:161)
	at org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:334)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putDown(BPlusTree.java:2499)
	at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doPut(BPlusTree.java:2252)
	... 11 more

Issue Links

• duplicates: IGNITE-11541 Dynamic columns and indexes can be lost after the cluster restart
• duplicates: IGNITE-12061 Silently fail while try to recreate already existing index with differ inline_size.