Minor Description
The following 2 tests frequently hang:
org.apache.ignite.internal.processors.igfs.IgfsTaskSelfTest#testTask
org.apache.ignite.internal.processors.igfs.IgfsTaskSelfTest#testTaskAsync
Also that happens on public TC.
Thread dump does not show any specific reason of stuck, but it frequently contains stack like this:
Thread name="test-runner-#298%igfs.IgfsTaskSelfTest%", id=364, state=RUNNABLE, blockCnt=14, waitCnt=25
at java.net.Inet6AddressImpl.getHostByAddr(Native Method)
at java.net.InetAddress$1.getHostByAddr(InetAddress.java:905)
at java.net.InetAddress.getHostFromNameService(InetAddress.java:590)
at java.net.InetAddress.getHostName(InetAddress.java:532)
at java.net.InetAddress.getHostName(InetAddress.java:504)
at o.a.i.i.processors.igfs.IgfsBlockLocationImpl.convertFromNodes(IgfsBlockLocationImpl.java:304)
at o.a.i.i.processors.igfs.IgfsBlockLocationImpl.<init>(IgfsBlockLocationImpl.java:101)
at o.a.i.i.processors.igfs.IgfsDataManager.splitBlocks(IgfsDataManager.java:895)
at o.a.i.i.processors.igfs.IgfsDataManager.affinity0(IgfsDataManager.java:862)
at o.a.i.i.processors.igfs.IgfsDataManager.affinity(IgfsDataManager.java:738)
at o.a.i.i.processors.igfs.IgfsImpl$18.call(IgfsImpl.java:1216)
at o.a.i.i.processors.igfs.IgfsImpl$18.call(IgfsImpl.java:1191)
at o.a.i.i.processors.igfs.IgfsImpl.safeOp(IgfsImpl.java:1679)
at o.a.i.i.processors.igfs.IgfsImpl.affinity(IgfsImpl.java:1191)
at o.a.i.igfs.mapreduce.IgfsTask.map(IgfsTask.java:116)
at o.a.i.igfs.mapreduce.IgfsTask.map(IgfsTask.java:85)
at o.a.i.i.processors.task.GridTaskWorker$2.call(GridTaskWorker.java:519)
at o.a.i.i.processors.task.GridTaskWorker$2.call(GridTaskWorker.java:517)
at o.a.i.i.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6509)
at o.a.i.i.processors.task.GridTaskWorker.body(GridTaskWorker.java:516)
at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:110)
at o.a.i.i.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:678)
at o.a.i.i.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:403)
at o.a.i.i.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:385)
at o.a.i.i.processors.igfs.IgfsImpl.executeAsync(IgfsImpl.java:1446)
at o.a.i.i.processors.igfs.IgfsImpl.executeAsync(IgfsImpl.java:1427)
at o.a.i.i.processors.igfs.IgfsImpl.execute(IgfsImpl.java:1375)
at o.a.i.i.processors.igfs.IgfsTaskSelfTest.testTask(IgfsTaskSelfTest.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:176)
at o.a.i.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:1760)
at o.a.i.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:118)
at o.a.i.testframework.junits.GridAbstractTest$4.run(GridAbstractTest.java:1698)
at java.lang.Thread.run(Thread.java:745)
measurements of method
org.apache.ignite.internal.processors.igfs.IgfsBlockLocationImpl#convertFromNodes ,
duration show the following:
:
...
convertFromNodes Took: 39 ms
convertFromNodes Took: 34 ms
convertFromNodes Took: 40 ms
convertFromNodes Took: 32 ms
convertFromNodes Took: 39 ms
convertFromNodes Took: 32 ms
convertFromNodes Took: 32 ms
convertFromNodes Took: 37 ms
convertFromNodes Took: 31 ms
convertFromNodes Took: 31 ms
convertFromNodes Took: 5067 ms
convertFromNodes Took: 33 ms
convertFromNodes Took: 31 ms
convertFromNodes Took: 137 ms
convertFromNodes Took: 33 ms
convertFromNodes Took: 30 ms
convertFromNodes Took: 41 ms
convertFromNodes Took: 35 ms
convertFromNodes Took: 136 ms
convertFromNodes Took: 71 ms
convertFromNodes Took: 5037 ms
convertFromNodes Took: 15056 ms
convertFromNodes Took: 37 ms
That is, addresses calculation sometimes takes as long as 15 seconds.
Simple caching of the addresses proven to fix the issue.