Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
4.3.5
-
None
-
httpcore-4.3.2
httpclient-4.3.5
Linux + Tomcat
Description
I have Apache HTTP client setup for 30 second connection timeout and 30 second read timeout.
In rare cases, due to a firewall issue, my outgoing connections are occasionally "blackholed" in the sense that the TCP connection remains up but absolutely zero TCP packets are received from the peer. So from the local side it just appears that the TCP connection is idle (outside the firewall the TCP connection may be closed, but my client can't see that inside the firewall).
If this happens at the wrong time during SSL connection setup, the connection hangs forever, even though it should be subject to an overall 30 second timeout.
Here's a stack trace showing a thread that is indefinitely stuck in this scenario:
"webClientTaskExecutor-3" prio=10 tid=0x00007fab3428a800 nid=0x4196 runnable [0x00007fab1647a000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at sun.security.ssl.InputRecord.readFully(InputRecord.java:442) at sun.security.ssl.InputRecord.read(InputRecord.java:480) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927) - locked <0x00000007a9472180> (a java.lang.Object) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) - locked <0x00000007a9472298> (a java.lang.Object) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:275) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:254) at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:123) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:318) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ...
An obvious workaround is to configure TCP keep-alive; I have not tried this yet.
However, the respect for the connect timeout should be bullet-proof, no matter what happens on the network.