[HTTPCORE-685] I/O dispatch threads spin after lots of TLSv1.3 connections

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 4.4.13, 4.4.14
• Component/s: HttpCore NIO
• Environment: java-11-openjdk-11.0.12.0.7-0.el8_4.x86_64 (11.0.2.12+7-LTS on Centos 8).
• Assignee: Unassigned
• Reporter: Chris Ridd
• Created: 09/Sep/21 11:04
• Updated: 19/Sep/21 10:05
• Resolved: 19/Sep/21 10:05

Description

When I run a Qualys VMDR scan against your example NHttpFileServer server, which is adjusted to only allow TLSv1.3, I see the two I/O-dispatch threads spinning, and a number of sockets (~40) never closing.

The adjusted example code is attached: AsyncExample.java

The I/O dispatch threads are consistently inside SSLIOSession.doHandshake() and handshaking is true and HandshakeStatus is NOT_HANDSHAKING. The threads seem to enter this code for every "bad" connection, and then repeat it all over again making no progress and keeping the sockets open.

As far as I see - the Qualys tests are sadly not open source - Qualys is just doing ~ 2000 HTTPS GET requests with different paths, apparently looking for known insecure applications. It is unclear what's special about the failing connections.

If I run the SyncExample.java server instead, the scan completes successfully.

If you can suggest any instrumentation that will gain some insights on this problem, I will be happy to assist.