Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
4.5.2
-
None
-
Security Level: Public (Anyone can view this level - this is the default.)
-
None
Description
A GetMethod request is created and used to initially connect to the web server where the template resides. It uses this connection to get the size and content-type, but as of commit 6de4dcb it doesn't use this connection to download. It creates a separate connection to download the template via a URL object, while the initial connection is left hanging with a body to read. Then at the very end, in a finally clause, it calls request.releaseConnection() for the original connection. In some of our environments this final close hangs, and seems to be correlated with the size of the template (how long it took to download) and the firewalls in between the SSVM and webserver, implying that the idle connection is timed out at some layer and can't be released. This may further trickle down into issues with the deprecated Commons HttpClient, but we believe the core issue is letting that initial connection go stale.
There is no justification in the commit notes for lines 236-238. We have tested rolling back just those lines (along with adjusting the imports to match), and it doesn't seem to affect speed or function and fixes the symptoms.
https://github.com/apache/cloudstack/commit/6de4dcb10a076671acc0f7486d493e914b59d1d4
You can verify the double connection via netstat, while a template is downloading you should see two connections, one with a full receive queue waiting for something to read the HTTP body, and one with an empty receive queue (the active one).