Details
Description
Log excerpt:
[06:28:44,643][SEVERE][tcp-disco-msg-worker-[f3d85cf4 10.114.158.249:47500 crd]-#2%ignite-instance-cfa2d711-f2b7-4572-84f3-b880e25aa9e4%-#87%ignite-instance-cfa2d711-f2b7-4572-84f3-b880e25aa9e4%][TcpDiscoverySpi] TcpDiscoverSpi's message worker thread failed abnormally. Stopping the node in order to prevent cluster wide instability. class org.apache.ignite.IgniteException: Arithmetic operation resulted in an overflow. at org.apache.ignite.internal.processors.platform.callback.PlatformCallbackUtils.consoleWrite(Native Method) at org.apache.ignite.internal.processors.platform.callback.PlatformCallbackGateway.consoleWrite(PlatformCallbackGateway.java:1203) at org.apache.ignite.internal.processors.platform.dotnet.PlatformDotNetConsoleStream.write(PlatformDotNetConsoleStream.java:44) at java.base/java.io.PrintStream.write(PrintStream.java:559) at java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:233) at java.base/sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:312) at java.base/sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) at java.base/java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:181) at java.base/java.io.PrintStream.write(PrintStream.java:606) at java.base/java.io.PrintStream.print(PrintStream.java:745) at org.apache.ignite.internal.util.IgniteUtils.quiet(IgniteUtils.java:4972) at org.apache.ignite.internal.util.IgniteUtils.quietAndWarn(IgniteUtils.java:4665) at org.apache.ignite.internal.processors.failure.FailureProcessor.process(FailureProcessor.java:174) at org.apache.ignite.internal.processors.failure.FailureProcessor.process(FailureProcessor.java:156) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1703) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1693) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:232) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:299) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$1(ServerImpl.java:2973) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:8050) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:3089) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7988) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
PlatformCallbackUtils.consoleWrite calls Callbacks.ConsoleWrite in .NET, and "Arithmetic operation resulted in an overflow." is from OverflowException.
We must be doing something wrong with JNI string conversion.
- Add try-catch, ignore exceptions in ConsoleWrite callback
- Investigate whether unicode characters or long strings are handled incorrectly
Workaround
Disable Java console redirect:
IgniteConfiguration.RedirectJavaConsoleOutput = false;