[HTTPCORE-750] H2ConnPool creates too many conns under high load at initialization time

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.2.1
• Fix Version/s: 5.2.2
• Component/s: HttpCore
• Environment: Linux / Ubuntu / openjdk version "1.8.0_292"
• Assignee: Unassigned
• Reporter: Gaojie Liu
• Created: 01/Jun/23 23:22
• Updated: 14/Jun/23 09:53
• Resolved: 04/Jun/23 18:11

Description

We are using httpcore5 5.2.1 and we observed that at application startup time, httpclient5 could create more than 1 conn per endpoint and in some cases, too many connections caused OOM issue since httpclient5 creates a pair of input/output buffer per conn.

The regression is introduced by HTTPCORE-750, which made this change:

https://github.com/apache/httpcomponents-core/commit/14caf43eae407c544161c7f92329e8beb42a3534

 

 if (poolEntry.session != null) {
                callback.completed(poolEntry.session);
            } else {
                poolEntry.requestQueue.add(callback);
                if (poolEntry.sessionFuture != null && poolEntry.sessionFuture.isDone()) {
                    poolEntry.sessionFuture = null;
                } 

When poolEntry.sessionFuture.isDone() is true, the connection is already ready, but the existing logic will abandon it and create a new one, and under high load, this logic could create a lot of conns per endpoint, which consumes a lot of memory.

 

The proposed fix:

if (poolEntry.session != null) {
                callback.completed(poolEntry.session);
            } else {
                poolEntry.requestQueue.add(callback);
                if (poolEntry.sessionFuture != null && poolEntry.sessionFuture.isDone()) {
                    // Check whether we should recreate a new conn or not
                    try {
                        poolEntry.session = poolEntry.sessionFuture.get();
                        while (true) {
                            final FutureCallback<IOSession> pendingCallback = poolEntry.requestQueue.poll();
                            if (pendingCallback != null) {
                                pendingCallback.completed(poolEntry.session);
                            } else {
                                break;
                            }
                        }
                    } catch (final Exception e) {
                        poolEntry.sessionFuture = null;
                    }
                } 

I am planning to cut a PR for this.

 

Issue Links

• is a clone of: HTTPCORE-683 H2ConnPool bug (stuck poolEntry)