Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7890

Invalidate ClusterConnectionState cache for a broker if the hostname of the broker changes.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.2.0, 2.1.1
    • Component/s: network
    • Labels:
      None

      Description

      We've ran into a similar issue as this ticket: https://issues.apache.org/jira/projects/KAFKA/issues/KAFKA-7755

      The fix for KAFKA-7755 doesn't work for this case as the hostname is not updated when resolving the addresses.

      `ClusterConnectionStates::connecting` method makes an assumption that broker ID will always map to same hostname. In our case, when a broker is terminated in AWS, it is replaced by a different instance under the same broker ID. 

      In this case, the consumer fails to connect to the right host when the broker ID returns to the cluster. For example, we see the following line in DEBUG logs:

      Initiating connection to node 100.66.7.94:7101 (id: 1 rack: us-east-1c) using address /100.66.14.165
      

      It tries to connect to the new broker instance using the wrong (old) IP address.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rsivaram Rajini Sivaram
                Reporter:
                markcho Mark Cho
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: