[KAFKA-17959] Avoid Reverse DNS Lookup for IP-Based SSL Authentication in Kraft Mode

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.6.0, 3.7.0, 3.7.1
• Fix Version/s: 3.8.0
• Component/s: kraft
• Assignee: Gaurav Narula
• Reporter: Arushi Helms
• Created: 07/Nov/24 22:33
• Updated: 13/Nov/24 22:01
• Resolved: 13/Nov/24 22:01

Description

We have encountered an issue with Kafka's Kraft mode where reverse DNS lookups are being performed unnecessarily during SSL authentication between controllers and between brokers and controllers, despite using IP addresses for communication.

In our Kafka setup, we are using IP addresses for communication and have configured certificates with IP addresses in the Subject Alternative Name (SAN). However, when the controller tries to establish SSL connections with other controllers or brokers, it attempts a reverse DNS lookup on the IP address (e.g., 10.87.170.83), which causes SSL handshake failures due to the mismatch between the resolved hostname and the IP address in the certificate.

The issue arises even though the certificate contains the IP in the SAN and should not require a reverse DNS lookup. This unnecessary lookup introduces delays and inconsistencies, especially in environments where DNS resolution is not required or reliable (e.g., in private networks).

Affected Scenarios:

1. Broker-to-Controller Communication: The broker fails to authenticate with the controller because the reverse DNS lookup of the controller's IP address does not match the expected DNS name in the certificate.
2. Controller-to-Controller Communication: Controllers also fail to authenticate with each other due to similar reverse DNS lookup issues.

Current Behavior:

• Kafka's SSL handshake fails when using IPs for communication, with errors like

  No subject alternative DNS name matching <resolved hostname> found

  due to reverse DNS lookup mismatches.

• The controller attempts reverse DNS lookups even when the connection is established using IP addresses directly.

Expected Behavior:

• Kafka should use the IP address directly for SSL engine creation and authentication when IPs are provided for communication, without performing a reverse DNS lookup.
• SSL hostname verification should match the IP address in the SAN of the certificate, not a resolved DNS name.

Request:

• Please address the issue by ensuring that Kafka does not perform reverse DNS lookups for SSL authentication when IP addresses are explicitly provided for communication.
• This behavior should be consistent across all Kafka components (brokers and controllers) in Kraft mode.

 

Old ticket with similar issue for reference:
https://issues.apache.org/jira/browse/KAFKA-5051