Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7755

Kubernetes - Kafka clients are resolving DNS entries only one time

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.2.0, 2.1.1
    • Component/s: clients
    • Labels:
      None
    • Environment:
      Kubernetes

      Description

      Introduction
      Since 2.1.0 Kafka clients are supporting multiple DNS resolved IP addresses if the first one fails. This change has been introduced by https://issues.apache.org/jira/browse/KAFKA-6863. However this DNS resolution is now performed only one time by the clients. This is not a problem if all brokers have fixed IP addresses, however this is definitely an issue when Kafka brokers are run on top of Kubernetes. Indeed, new Kubernetes pods will receive another IP address, so as soon as all brokers will have been restarted clients won't be able to reconnect to any broker.

      Impact
      Everyone running Kafka 2.1 or later on top of Kubernetes is impacted when a rolling restart is performed.

      Root cause
      Since https://issues.apache.org/jira/browse/KAFKA-6863 Kafka clients are resolving DNS entries only once.

      Proposed solution
      In https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/ClusterConnectionStates.java#L368 Kafka clients should perform the DNS resolution again when all IP addresses have been "used" (when index is back to 0)

        Attachments

        1. pom.xml
          30 kB
          Chris Bogan

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                loic.monney Loïc Monney
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: