[KAFKA-8978] When TLS is applied, broker sends the response frame length in separate TLS packet

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 2.2.1
• Component/s: core
• Assignee: Unassigned
• Reporter: Piotr Smolinski
• Created: 04/Oct/19 11:18
• Updated: 04/Oct/19 21:07

Description

The issue was discovered tracking the network dumps.

Each frame in Kafka wire protocol is prefixed with the 32-bit integer which contains  remaining frame length. When unencrypted TCP is used (PLAINTEXT), the 4 octets are sent inside the same TCP/IP packet as the rest of the frame (at least as long as the whole frame fits). When TLS is enabled, the buffer is flushed immediately after the length is written which results in 33-octets length TLS packet and 101-octets length wire frame carrying just 4 octets of data.

In the attachments there is tcp dump and captured encryption keys executing metadata query with kafkacat. 

The capture was analysed with Wireshark built from the source code. The support for Kafka 2.3 and Kafka over TLS dissection was merged into the master branch, but it has not been released yet.