[HADOOP-15447] SwiftRestClient does not propertly close connections

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.9.0, 2.6.5, 2.7.4, 3.0.2
• Component/s: fs/swift
• Assignee: Unassigned
• Reporter: Jelmer Kuperus
• Created: 04/May/18 07:32
• Updated: 04/May/18 07:33

Description

On production the output of netstat -a showed a large number of connections to swift hosts in the  CLOSE_WAIT and ESTABLISHED state

The reason is that while the SwiftRestClient calls releaseConnection, because HTTP 1.1 is used, the connection is not closed by the http client. Because it could  pipeline further requests to the same host:port in the same connection.

One way to remedy this is to add a "Connection: close" header

This article further explains the issue : https://doc.nuxeo.com/blog/using-httpclient-properly-avoid-closewait-tcp-connections/