[HTTPCORE-600] SocketTimeoutException when remote server closes connection after payload has been delivered

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.4.10, 4.4.11, 4.4.12
• Fix Version/s: 4.4.13, 5.0-beta9
• Component/s: HttpCore NIO
• Assignee: Oleg Kalnichevski
• Reporter: Brian B
• Created: 12/Sep/19 15:11
• Updated: 22/Sep/19 14:17
• Resolved: 16/Sep/19 20:51

Description

Initially I created this under CXF (CXF-8113), but I think the consensus is it is an NIO issue.  The description from that Jira edited to remove CXF-specific changes:

When using the Asynchronous Client HTTP Transport to handle soap web services, we get a SocketTimeout exception when a remote server closes the connection after the payload has been delivered.

The problem occurs when the client receives a TLS close_notify after the payload has been received.  The payload must be larger than the allocated byte buffer receiving the decrypted payload (~16K in the default case). This results in decrypted data being available in the httpcore-nio library, but not immediately consumed. 

When these conditions are met, the following section of code is invoked:
org.apache.cxf.transport.http.asyncclient.SharedInputBuffer#consumeContent:111:

if (!this.buffer.hasRemaining() && this.ioctrl != null && !this.endOfStream) {
    this.ioctrl.suspendInput();
}

The suspension of input when combined with the connection close and the following change introduced in httpcore-nio 4.4.10 (and it's revised 4.4.11 version here) truly removes the READ EventMask from the underlying IOSessionImpl:
org.apache.http.nio.reactor.ssl.SSLIOSession#updateEventMask:402

if (this.endOfStream && (this.appBufferStatus == null || !this.appBufferStatus.hasBufferedInput())) {
    newMask = newMask & ~EventMask.READ;
}

Once this happens, requests for input by SharedInputBuffer#waitForData(int) enter the updateEventMask method as expected, but the new httpcore-nio code above prevents the read operation from being reenabled despite the remainder of the decrypted payload being available in the SSLIOSession inPlain buffer.  The call ultimately fails with a SocketTimeoutException.

I created a unit test to demonstrate the failure and attached it to the JIRA.  Given the complexity of the code and multiple buffers at play, I have not been able to come up with a fix beyond modifying the SSLIOSession code above to account for the buffered decrypted content.  It seems in all cases where appBufferStatus.hasBufferedInput() is queried, inPlain.hasData() is also queried, with the exception of the updateEventMask code above.

Please let me know if there's anything else you need from me.

 

Issue Links

• is related to: CXF-8113 SocketTimeoutException when remote server closes connection after payload has been delivered