Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-16799

NetworkClientDelegate is not backing off if the node is not found

    XMLWordPrintableJSON

Details

    Description

      When performing stress testing, I found that AsycnKafkaConsumer's network client delegate isn't backing off if the node is not ready, causing a large number of: 

       358 [2024-05-20 22:59:02,591] DEBUG [Consumer clientId=consumer.7136899e-0c20-4ccb-8ba3-497e9e683594-0, groupId=consumer-groups-test-5] Node is not ready, handle the request in the next event loop: node=b4-pkc-devcmkz697.us-west-2.aws.devel.cpd     ev.cloud:9092 (id: 2147483643 rack: null), request=UnsentRequest{requestBuilder=ConsumerGroupHeartbeatRequestData(groupId='consumer-groups-test-5', memberId='', memberEpoch=0, instanceId=null, rackId=null, rebalanceTimeoutMs=100000, subscri     bedTopicNames=[_kengine-565-test-topic8081], serverAssignor=null, topicPartitions=[]), handler=org.apache.kafka.clients.consumer.internals.NetworkClientDelegate$FutureCompletionHandler@139a8761, node=Optional[b4-pkc-devcmkz697.us-west-2.aws     .devel.cpdev.cloud:9092 (id: 2147483643 rack: null)], timer=org.apache.kafka.common.utils.Timer@649fffad} (org.apache.kafka.clients.consumer.internals.NetworkClientDelegate:169) 

      show up in the log.

      What should have happened is: 1. node is not ready 2. exponential back off 3. retry

      Attachments

        Activity

          People

            pnee Philip Nee
            pnee Philip Nee
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: