Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7954

When broker IP addresses change client never resolves new addresses to fetch metadata

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.1.0
    • Fix Version/s: None
    • Component/s: clients
    • Labels:
      None

      Description

      I'm running Kafka on AWS autoscaling group. Currently we are having infrastructure immutable and we have no fixed IP addresses. 

      When we recreate out Kafka cluster all brokers get new IP addresses, even in case when all nodes die horrible death simultaneously. 

      Looking at the code IP addresses are resolved and added ClusterConnectionStates class line 119: 

       

      nodeState.put(id, new NodeConnectionState(ConnectionState.CONNECTING, now,
      this.reconnectBackoffInitMs, ClientUtils.resolve(host, clientDnsLookup)));

      At this point ClientUtils.resolve resolves IP addreses. If brokers receive new IP addresses each subsequent metadata fetch will fail because none of the nodes will be addressable by IP addresses that where resolved on previous metadata fetch. 

       

      Since addresses list will never be evicted old IP addresses will stay there forever. 

       

       

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rpofuk@gmail.com Robert Pofuk
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: