soTimeout transport property- Incorrect documentation (or code, you pick)

• Key: AMQ-1860
• Type: Bug
• Status: Resolved
• Resolution: Fixed
• Priority: Major
• Assignee: Rob Davies
• Reporter: Filip Hanik
• Component/s: Documentation, Broker
• Affects Version/s: 5.0.0, 5.1.0
• Fix Version/s: 5.2.0

Description

The documentation for TCP transport, shows soTimeout, connectionTimeout and socketBufferSize as properties without a prefix.
The code in TcpTransportServer.java doesn't have the support for these properties.
hence setting a URL to tcp://192.168.3.3:61616?soTimeout=180000 does nothing, and defaults to soTimeout=0 (which is a bad value due to how sockets get closed, and can cause the system to hang on a socketWrite call, causing the rest of the server to hang, but I can explain that on the dev lists for those interested)

Bug description: Transport properties are incorrectly document or implemented

There are two solutions to this bug:

1. The preferred, would be to fix the documentation
http://activemq.apache.org/tcp-transport-reference.html
the mentioned options will work if they are prefixed with "transport."
so the URL would be
tcp://192.168.3.3:61616?transport.soTimeout=180000

2. Add the properties with their getters/setters into TcpTransportServer.java and then during the handleSocket method add them to the options map

Please note, this bug refers to the soTimeout setting on the socket that gets created on the server when a producer/consumer connects in using TCP

best
Filip

Comments

[Filip Hanik]

As a side note, and not relevant to this issue, let me explain why soTimeout=0 is a bad default value

Lets say you have a connection between a consumer and the AMQ server, that connections gets terminated.

When a socket is closed, it doesn't really mean its closed - this is explained in:
http://java.sun.com/j2se/1.5.0/docs/guide/net/articles/connection_release.html

On the server the following will happen

yy.yy.yy.yy.61616 xx.xx.xx.xx.44674 49152 0 49330 0 CLOSE_WAIT

And inside of the AMQ you'll see the following stack trace
"ActiveMQ Transport: tcp:///xx.xx.xx.xx:44674" daemon prio=4 tid=0x002ebc88 nid=0x2a runnable [0xd10ff000..0xd10ffc70]
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:105)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:154)
at org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:157)

• locked <0xd8775fc0> (a org.apache.activemq.transport.InactivityMonitor$2)
  at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:82)
  at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:91)
  at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
• locked <0xd8775da8> (a java.lang.Object)
  at org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1151)
  at org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:766)
  at org.apache.activemq.broker.TransportConnection.dispatchSync(TransportConnection.java:727)

please remember, the client is no longer connected, and is no longer reading data. The server now has a CLOSE_WAIT connection and a thread that is stuck in this state.
The socketWrite0 will only return after a TCP packets has moved on the connection (which it wont since the client is no longer present) or until it times out (which it wont since soTimeout is 0)

As you can see this thread is also holding a bunch of locks, some of these interfer with other server threads, causing the system to display systems of being hung. The only work around ar this point is to restart the AMQ server.

Hope this makes sense. My recommendation would be to have a >0 soTimeout value as default, (and the property name be documented properly of course )

best
Filip

[Filip Hanik]

Looks like the problems go further than just the documentation being incorrect. On the server, it the soTimeout setting wont take into effect unless you use transport.soTimeout property.
However, trying to apply this setting to a client URL, the system bombs out
Caused by: java.lang.IllegalArgumentException: Invalid connect parameters: {transport.soTimeout=180000}
at org.apache.activemq.transport.TransportFactory.doConnect(TransportFactory.java:133)
at org.apache.activemq.transport.TransportFactory.doConnect(TransportFactory.java:47)
at org.apache.activemq.transport.TransportFactory.connect(TransportFactory.java:76)
at org.apache.activemq.ActiveMQConnectionFactory.createTransport(ActiveMQConnectionFactory.java:235)

then it would probably be better to just allow the property soTimeout, but it looks like the entire socket property settings need to have a look over, since there is no consistent way of setting these properties. And because the soTimeout property is not set, the whoe server locks up if there is a client that is deciding to stop receiving data.

[Filip Hanik]

Example of how to make the soTimeout property work on the server

[Filip Hanik]

Example of how to make the soTimeout work for the server, one must of course verify that the client is picking up the option too

[Rob Davies]

Thanks Filip - applied the patch in SVN revision 685808

[Filip Hanik]

Rob, looks like I wrote the patch with my head up my rear,

this

+ options.put("soTimeout", Integer.valueOf(soTimeout));
+ options.put("soTimeout", Integer.valueOf(soTimeout));
+ options.put("connectionTimeout", Integer.valueOf(socketBufferSize));

should be

+ options.put("soTimeout", Integer.valueOf(soTimeout));
+ options.put("socketBufferSize", Integer.valueOf(socketBufferSize));
+ options.put("connectionTimeout", Integer.valueOf(connectionTimeout));

[Filip Hanik]

See previous note, copy paste error in the included patch

[Rob Davies]

no problem! - updated in SVN revision 691129