[IGNITE-7207] Exchange worker dies if index contains non existent field

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.3
• Assignee: Stanislav Lukyanov
• Reporter: Valentin Kulichenko
• Created: 15/Dec/17 02:55
• Updated: 17/Jul/18 10:04
• Resolved: 17/Jul/18 10:04

Description

If QueryEntity contains an index with a non existent field, createCache first throws this exception which is correct:

[2017-12-14 18:51:23,627][ERROR][exchange-worker-#42%null%][CacheAffinitySharedManager] Failed to initialize cache. Will try to rollback cache start routine. [cacheName=test]
class org.apache.ignite.IgniteCheckedException: Field not found: a
	at org.apache.ignite.internal.processors.query.QueryIndexDescriptorImpl.addField(QueryIndexDescriptorImpl.java:124)
	at org.apache.ignite.internal.processors.query.QueryUtils.createIndexDescriptor(QueryUtils.java:631)
	at org.apache.ignite.internal.processors.query.QueryUtils.processIndex(QueryUtils.java:648)
	at org.apache.ignite.internal.processors.query.QueryUtils.processIndexes(QueryUtils.java:584)
	at org.apache.ignite.internal.processors.query.QueryUtils.processBinaryMeta(QueryUtils.java:542)
	at org.apache.ignite.internal.processors.query.QueryUtils.typeForQueryEntity(QueryUtils.java:438)
	at org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart0(GridQueryProcessor.java:687)
	at org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:836)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1111)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1816)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:751)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:882)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:588)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2278)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
	at java.lang.Thread.run(Thread.java:745)
Exception in thread "main" javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Field not found: a
	at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1327)
	at org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2817)
	at Test.main(Test.java:36)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: class org.apache.ignite.IgniteCheckedException: Field not found: a
	at org.apache.ignite.internal.processors.query.QueryIndexDescriptorImpl.addField(QueryIndexDescriptorImpl.java:124)
	at org.apache.ignite.internal.processors.query.QueryUtils.createIndexDescriptor(QueryUtils.java:631)
	at org.apache.ignite.internal.processors.query.QueryUtils.processIndex(QueryUtils.java:648)
	at org.apache.ignite.internal.processors.query.QueryUtils.processIndexes(QueryUtils.java:584)
	at org.apache.ignite.internal.processors.query.QueryUtils.processBinaryMeta(QueryUtils.java:542)
	at org.apache.ignite.internal.processors.query.QueryUtils.typeForQueryEntity(QueryUtils.java:438)
	at org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart0(GridQueryProcessor.java:687)
	at org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:836)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1111)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1816)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:751)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:882)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:588)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2278)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
	at java.lang.Thread.run(Thread.java:745)

However, right after that exchange worker gets stuck with the following:

[2017-12-14 18:51:23,633][ERROR][exchange-worker-#42%null%][GridDhtPartitionsExchangeFuture] Failed to reinitialize local partitions (preloading will be stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], discoEvt=DiscoveryCustomEvent [customMsg=DynamicCacheChangeBatch [id=c2c45185061-3cd3ec99-6332-4ae0-b9b1-d88b2d101179, reqs=[DynamicCacheChangeRequest [cacheName=test, hasCfg=true, nodeId=27a25ec6-5852-43cb-ae0d-fe6f00fd529a, clientStartOnly=false, stop=false, destroy=false]], exchangeActions=ExchangeActions [startCaches=[test], stopCaches=null, startGrps=[test], stopGrps=[], resetParts=null, stateChangeRequest=null], startCaches=false], affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=27a25ec6-5852-43cb-ae0d-fe6f00fd529a, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 192.168.1.64, 192.168.56.1, 2602:304:cdc1:4360:10dd:59f2:3930:a688, 2602:304:cdc1:4360:f072:3981:da90:2c92], sockAddrs=[/2602:304:cdc1:4360:10dd:59f2:3930:a688:47500, /192.168.1.64:47500, /192.168.56.1:47500, /0:0:0:0:0:0:0:1:47500, /2602:304:cdc1:4360:f072:3981:da90:2c92:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1513306281799, loc=true, ver=2.2.0#19691231-sha1:00000000, isClient=false], topVer=1, nodeId8=27a25ec6, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1513306283580]], nodeId=27a25ec6, evt=DISCOVERY_CUSTOM_EVT]
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(ArrayList.java:653)
	at java.util.ArrayList.get(ArrayList.java:429)
	at org.apache.ignite.internal.processors.cache.CacheGroupContext.singleCacheContext(CacheGroupContext.java:347)
	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLocalPartition.<init>(GridDhtLocalPartition.java:187)
	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.createPartition(GridDhtPartitionTopologyImpl.java:741)
	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.initPartitions0(GridDhtPartitionTopologyImpl.java:358)
	at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.beforeExchange(GridDhtPartitionTopologyImpl.java:542)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1015)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:651)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2278)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
	at java.lang.Thread.run(Thread.java:745)

Reproducer attached.

Issue Links

• duplicates: IGNITE-1094 Ignite.createCache(CacheConfiguration) hangs if some exception occurs during cache initialization
• relates to: IGNITE-5026 getOrCreateCaches() hangs if any exception in GridDhtPartitionsExchangeFuture.init()
• relates to: IGNITE-1094 Ignite.createCache(CacheConfiguration) hangs if some exception occurs during cache initialization