[HTTPCLIENT-2133] "Connection pool shut down" Exceptions occur after Error such as OOM is thrown while making an HTTP request

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 4.5.6, 4.5.7, 4.5.8, 4.5.9, 4.5.10, 4.5.11, 4.5.12, 4.5.13, 5.0 Beta2
• Component/s: HttpClient (classic)
• Assignee: Unassigned
• Reporter: Gordon Daugherty
• Created: 28/Jan/21 01:16
• Updated: 31/Jan/21 09:06
• Resolved: 31/Jan/21 09:06

Description

The change made in HTTPCLIENT-1924 was an improvement but introduced a new problem.

Scenario:

Software initializes and begins using an HttpClient like so:
{{}}

    PoolingHttpClientConnectionManager cm = connectionManagerBuilder.build();    
    HttpClient client = HttpClientBuilder.create().setConnectionManager(cm).build();

    HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory(client);
    factory.setReadTimeout(clientConfiguration.getReadTimeoutMs());
    factory.setConnectTimeout(clientConfiguration.getConnectTimeoutMs());
    factory.setConnectionRequestTimeout(clientConfiguration.getTimeoutConnectionRequestMs());
    factory.setBufferRequestBody(bufferRequestResponse(clientConfiguration));
    
    RestTemplate restTemplate = new RestTemplate(factory);
    restTemplate.setErrorHandler(new HttpClientErrorHandlerSpringBridge(clientConfiguration.getErrorHandler()));
    restTemplate.setInterceptors(this.interceptors);
    restTemplate.setMessageConverters(this.messageConverters);
    return restTemplate;

Note that this }}{{PoolingHttpClientConnectionManager instance is used by only one HttpClient. It is not being shared. 

Later while using the RestTemplate and in turn this HttpClient the software experiences an OOM Error. Perhaps caused by an unexpectedly large response payload. When that error occurs near the end of class org.apache.http.impl.execchain.MainClientExec's "execute" method it gets handled by this code:

        } catch (final Error error) {
            connManager.shutdown();
            throw error;
        }

The problem is that while the software that's using the affected HttpClient instance knows than an OOM Error occurred and presumably handles that it can't know that a pool was shut down and that the HttpClient is now unusable.

Consider modifying the solution that was applied in HTTPCLIENT-1924 to instead mimic how exceptions are handled in class MainClientExec:

        } catch (final Error error) {
// REMOVE:            connManager.shutdown();
            connHolder.abortConnection();
            if (proxyAuthState.isConnectionBased()) {
                proxyAuthState.reset();
            }
            if (targetAuthState.isConnectionBased()) {
                targetAuthState.reset();
            }
            throw error;
        }

This alternate solution will release the connection back to the pool in a state where the pool will not attempt to reuse it. This prevents leaks without preventing continued use of the HttpClient and its supporting connection pool.