[CASSANDRA-12513] IOException (No such file or directory) closing MessagingService's server socket (locally)

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 3.0.13, 3.11.0, 4.0-alpha1, 4.0
• Severity: Low
• Assignee: Robert Stupp
• Reporter: Robert Stupp

Authors

Robert Stupp

Reviewers

Michael Kjellman

• Created: 22/Aug/16 06:43
• Updated: 15/May/20 08:04
• Resolved: 08/Mar/17 13:00

Description

Sometimes the RemoveTest fails with the following exception. It's not related to the test itself.

The exception is risen in ServerSocketChannelImpl.implCloseSelectableChannel where it checks that a thread ID is non-zero. The thread instance field is set inside its accept and poll methods. It looks like this is caused by some race condition - i.e. stopping in debugger at certain points prevents it from being triggered.
I could not find any misuse in the code base - but want to document this issue.

No difference between 8u92 and 8u102

INFO  [ACCEPT-/127.0.0.1] 2016-08-22 08:35:16,606 ?:? - MessagingService has terminated the accept() thread

java.io.IOError: java.io.IOException: No such file or directory

	at org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:914)
	at org.apache.cassandra.service.RemoveTest.tearDown(RemoveTest.java:103)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
	at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
Caused by: java.io.IOException: No such file or directory
	at sun.nio.ch.NativeThread.signal(Native Method)
	at sun.nio.ch.ServerSocketChannelImpl.implCloseSelectableChannel(ServerSocketChannelImpl.java:292)
	at java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(AbstractSelectableChannel.java:234)
	at java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:115)
	at sun.nio.ch.ServerSocketAdaptor.close(ServerSocketAdaptor.java:137)
	at org.apache.cassandra.net.MessagingService$SocketThread.close(MessagingService.java:1249)
	at org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:904)
	... 22 more

Issue Links

• relates to: SPARK-30385 WebUI occasionally throw IOException on stop()