[IGNITE-1666] Fallback for full scan query does not work

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: ignite-1.4
• Component/s: cache
• Assignee: Unassigned
• Reporter: Andrey N. Gura
• Created: 13/Oct/15 13:53
• Updated: 28/Jan/20 13:07
• Resolved: 28/Jan/20 13:07

Description

Fallback for full scan query doesn't work.

Steps to reproduce (all in one thread):

1. Start N nodes, create cache and populate some entries to cache.
2. Stop some node.
3. Get iterator on the cache and fetch all entries one by one.
4. Repeat from step 2.

On some iteration scan query fails with exception:

javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Query execution failed: GridCacheQueryBean [qry=GridCacheQueryAdapter [type=SCAN, clsName=null, clause=null, filter=null, part=null, incMeta=false, metrics=GridCacheQueryMetricsAdapter [minTime=0, maxTime=0, sumTime=0, avgTime=0.0, execs=0, completed=0, fails=0], pageSize=1024, timeout=0, keepAll=false, incBackups=false, dedup=false, prj=null, keepPortable=false, subjId=3026635f-2a79-4a59-83cb-9db427f45003, taskHash=0], rdc=null, trans=null]
	at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1614)
	at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:181)
	at org.apache.ignite.internal.processors.cache.CacheWeakQueryIteratorsHolder$WeakQueryFutureIterator.init(CacheWeakQueryIteratorsHolder.java:228)
	at org.apache.ignite.internal.processors.cache.CacheWeakQueryIteratorsHolder$WeakQueryFutureIterator.onHasNext(CacheWeakQueryIteratorsHolder.java:180)
	at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
	at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
	at org.apache.ignite.internal.processors.cache.distributed.CacheIteratorWhileRemovalSelfTest.doTest(CacheIteratorWhileRemovalSelfTest.java:135)
	at org.apache.ignite.internal.processors.cache.distributed.CacheIteratorWhileRemovalSelfTest.testFullScanIterator(CacheIteratorWhileRemovalSelfTest.java:91)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at junit.framework.TestCase.runTest(TestCase.java:176)
	at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:1658)
	at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:112)
	at org.apache.ignite.testframework.junits.GridAbstractTest$4.run(GridAbstractTest.java:1596)
Caused by: class org.apache.ignite.IgniteCheckedException: Query execution failed: GridCacheQueryBean [qry=GridCacheQueryAdapter [type=SCAN, clsName=null, clause=null, filter=null, part=null, incMeta=false, metrics=GridCacheQueryMetricsAdapter [minTime=0, maxTime=0, sumTime=0, avgTime=0.0, execs=0, completed=0, fails=0], pageSize=1024, timeout=0, keepAll=false, incBackups=false, dedup=false, prj=null, keepPortable=false, subjId=3026635f-2a79-4a59-83cb-9db427f45003, taskHash=0], rdc=null, trans=null]
	at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.checkError(GridCacheQueryFutureAdapter.java:267)
	at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.internalIterator(GridCacheQueryFutureAdapter.java:276)
	at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:171)
	... 14 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to execute query on node [query=GridCacheQueryBean [qry=GridCacheQueryAdapter [type=SCAN, clsName=null, clause=null, filter=null, part=null, incMeta=false, metrics=GridCacheQueryMetricsAdapter [minTime=0, maxTime=0, sumTime=0, avgTime=0.0, execs=0, completed=0, fails=0], pageSize=1024, timeout=0, keepAll=false, incBackups=false, dedup=false, prj=null, keepPortable=false, subjId=3026635f-2a79-4a59-83cb-9db427f45003, taskHash=0], rdc=null, trans=null], nodeId=203d0a36-8587-4a32-aa51-3332d126d002]
	at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.onPage(GridCacheQueryFutureAdapter.java:390)
	at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryFuture.onNodeLeft(GridCacheDistributedQueryFuture.java:146)
	at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$3.onEvent(GridCacheDistributedQueryManager.java:114)
	at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.notifyListeners(GridEventStorageManager.java:770)
	at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.notifyListeners(GridEventStorageManager.java:755)
	at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.record(GridEventStorageManager.java:295)
	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.recordEvent(GridDiscoveryManager.java:1949)
	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:2156)
	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:1989)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
	at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: Remote node has left topology: 203d0a36-8587-4a32-aa51-3332d126d002
	... 10 more

In this case query should be retried (as well as for partition scan query).

Test for reproducing could be found in PR https://github.com/apache/ignite/pull/149

Issue Links

• duplicates: IGNITE-12591 Scan query scans invalid partitions on unstable topology
• links to: GitHub Pull Request #149
• links to: GitHub Pull Request #6823
• links to: PR with test