[HTTPCLIENT-2007] Performance Issue with Pooled Connection Manager + Custom socket factory

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 4.5.9
• Component/s: HttpClient (classic)
• Assignee: Unassigned
• Reporter: Venu Kemthur
• Created: 07/Aug/19 04:29
• Updated: 07/Aug/19 17:32
• Resolved: 07/Aug/19 17:32

Description

We are using HttpClient 4.5.9 to connect to a service which expects Client Cert (MTLS)

Since we are using PoolingHttpClientConnectionManager, had to use the following mechanism to create and set a LayeredConnectionSocketFactory when instantiating PoolingHttpClientConnectionManager

ConnectionSocketFactory plainsf = <...>
LayeredConnectionSocketFactory sslsf = <...>
Registry<ConnectionSocketFactory> r = RegistryBuilder.<ConnectionSocketFactory>create()
        .register("http", plainsf)
        .register("https", sslsf)
        .build();

HttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(r);
HttpClients.custom()
        .setConnectionManager(cm)
        .build();

Tested this service with K6 load testing tool and see latency in seconds. Seems like it keeps growing from milliseconds (20ms) all the way to 8 seconds.

K6 in a minute could only do 45 QPS with this latency.

Sample run

data_received..............: 1.6 MB 27 kB/s
 data_sent..................: 6.9 MB 115 kB/s
 http_req_blocked...........: avg=8.92ms min=1µs med=3µs max=424.79ms p(90)=6µs p(95)=13µs
 http_req_connecting........: avg=1.42ms min=0s med=0s max=46.53ms p(90)=0s p(95)=0s
 http_req_duration..........: avg=2.1s min=57.25ms med=2.11s max=5.96s p(90)=4.1s p(95)=4.22s
 http_req_receiving.........: avg=71.79µs min=23µs med=55µs max=6.19ms p(90)=89µs p(95)=102µs
 http_req_sending...........: avg=25.37µs min=9µs med=20µs max=2.38ms p(90)=35µs p(95)=43µs
 http_req_tls_handshaking...: avg=7.49ms min=0s med=0s max=385.7ms p(90)=0s p(95)=0s
 http_req_waiting...........: avg=2.1s min=57.18ms med=2.11s max=5.96s p(90)=4.1s p(95)=4.22s
 http_reqs..................: 2721 45.349907/s
 iteration_duration.........: avg=2.11s min=64.04ms med=2.11s max=6.12s p(90)=4.11s p(95)=4.22s
 iterations.................: 2721 45.349907/s
 vus........................: 100 min=100 max=100
 vus_max....................: 100 min=100 max=100

Same code when replaced with JDK 11 HttpClient could do 600 QPS!

We are using this client in other scenarios (non-MTLS) and we have no issues in that path.

Is there a known issue with Pooling Connection Manager +SSLConnectionSocketFactory