[PROTON-2535] TLS library - false indication of user data in OpenSSL

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: proton-c-0.37.0
• Fix Version/s: proton-c-0.38.0
• Component/s: proton-c
• Environment: OpenSSL
• Assignee: Clifford Jansen
• Reporter: Clifford Jansen
• Created: 19/Apr/22 14:39
• Updated: 11/Nov/22 14:37
• Resolved: 01/Nov/22 06:07

Description

pn_tls_need_decrypt_output_buffers can falsely indicate the availability of user data.  For example if there is a handshake failure, BIO_pending can indicate the presence of bytes but BIO_read will return -1 and the appropriate error.

An application may be fooled into providing a decrypt output buffer that won't be immediately be returned after the next pn_tls_process() step, since no bytes will be read into it.