[HTTPCORE-710] Async client hangs if remote peer closes connection during TLS handshake

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 5.2-beta1
• Fix Version/s: 5.2-beta2
• Component/s: HttpCore
• Assignee: Unassigned
• Reporter: Ryan Schmitt
• Created: 24/Mar/22 03:50
• Updated: 07/Apr/22 13:26
• Resolved: 07/Apr/22 13:26

Description

I discovered a regression in the 5.2 async client while testing various failure modes for TLS negotiation. In this case, the failure condition is when the client sends a Client Hello and the server responds by immediately closing the connection (with a FIN in this case, not a RST). With client version 5.1.3, when this happened the client would simply throw an exception with the following cause:

        Caused by:
        org.apache.hc.core5.http2.impl.nio.ProtocolNegotiationException: Unexpected input
            at app//org.apache.hc.core5.http2.impl.nio.ClientHttpProtocolNegotiator.inputReady(ClientHttpProtocolNegotiator.java:165)
            at app//org.apache.hc.core5.reactor.ssl.SSLIOSession.decryptData(SSLIOSession.java:574)
            at app//org.apache.hc.core5.reactor.ssl.SSLIOSession.access$400(SSLIOSession.java:72)
            at app//org.apache.hc.core5.reactor.ssl.SSLIOSession$1.inputReady(SSLIOSession.java:172)
            at app//org.apache.hc.core5.reactor.InternalDataChannel.onIOEvent(InternalDataChannel.java:131)
            at app//org.apache.hc.core5.reactor.InternalChannel.handleIOEvent(InternalChannel.java:51)
            ... 5 more

Now, on the latest 5.2 nightly, no exception gets thrown at all; whereas SSLIOSession used to call into ClientHttpProtocolNegotiator#inputReady, it now ends up in AbstractHttp1IOEventHandler#inputReady and then AbstractHttp1StreamDuplexer#onInput, which placidly requests a graceful shutdown by following this code path:

        if (endOfStream && !inbuf.hasData()) {
            if (outputIdle() && inputIdle()) {
                requestShutdown(CloseMode.GRACEFUL);

However, even if I force the code to take the other path (shutdownSession(new ConnectionClosedException("Connection closed by peer"))), the client still just hangs forever. Something, presumably SSLIOSession itself, is forgetting to invoke the appropriate callback to fail the handshake in this situation.