[AMQ-6937] Recycling TCP/IP stack on z/OS causes an infinite error loop in transport server

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 5.15.3
• Component/s: Broker
• Assignee: Unassigned
• Reporter: Tomas Pavelka
• Created: 23/Mar/18 10:13
• Updated: 18/Apr/18 06:43

Description

The ActiveMQ transport servers (e.g. TcpTransportServer) run the socket accept (java.net.ServerSocket#accept) in an infinite loop. The accept call can repeatedly fail with an exception spinning the CPU at full speed an filling up logs quickly.

Here is an example of an exception that gets repeated indefinitely:

java.net.SocketException: EDC5122I Input/output error. (Accept failed)
    at java.net.ServerSocket.implAccept(ServerSocket.java:623)
    at java.net.ServerSocket.accept(ServerSocket.java:582)
    at org.apache.activemq.transport.tcp.TcpTransportServer.run(TcpTransportServer.java:351)
    at java.lang.Thread.run(Thread.java:785)

This is a common problem on z/OS because the pattern of running accept in a loop is used in many open source projects. For example, here is the same issue in Derby:

https://issues.apache.org/jira/browse/DERBY-5347

And here in Jetty:

https://github.com/eclipse/jetty.project/issues/283

Whenever the problem appears the socket becomes unusable. Would it be possible for ActiveMQ to allow to insert a custom org.apache.activemq.transport.TransportAcceptListener that would detect the problem and do a re-bind on the socket?