Major Description
There are a couple of issues with the handling of buffer overflow in SSLTransportLayer.
- netWriteBuffer is flipped immediately after wrap(), leaving the data ready for writing onto the socket channel. If netWriteBuffer is expanded because wrap() returns BUFFER_OVERFLOW, the expanded buffer needs to be un-flipped before expansion and flipped afterwards to leave the resulting buffer in the same state. The current implementation does not do this and hence the expanded buffer is not as expected.
- If handshakeUnwrap() returns a BUFFER_OVERFLOW because application buffer needs to be expanded, the current implementation expands the buffer and returns from the handshake() call. Since handshake data was already read from the network, this can result in the handshake never completing if no more data is received on the channel. handshakeUnwrap() should be invoked after application buffer expansion to process data that has already arrived.
I will submit a PR with fixes for these along with unit tests which demonstrate these issues. I am not sure it is easy to trigger buffer overflow/underflow in a system test. Since these code paths are currently untested, unit tests which mock these scenarios may be useful.
Attachments
Issue Links
- links to