Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-16637

AsyncKafkaConsumer removes offset fetch responses from cache too aggressively

    XMLWordPrintableJSON

Details

    Description

      I want to test next generation of the consumer rebalance protocol  (https://cwiki.apache.org/confluence/display/KAFKA/The+Next+Generation+of+the+Consumer+Rebalance+Protocol+%28KIP-848%29+-+Early+Access+Release+Notes)

       

      However, it does not works well. 

      You can check my condition.

       

      Docker-compose.yaml

      https://github.com/chickenchickenlove/kraft-test/blob/main/docker-compose/docker-compose.yaml

       

      Consumer Code

      https://github.com/chickenchickenlove/kraft-test/blob/main/src/main/java/org/example/Main.java

       

      Consumer logs

      [main] INFO org.apache.kafka.common.telemetry.internals.KafkaMetricsCollector - initializing Kafka metrics collector
      [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 3.7.0
      [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: 2ae524ed625438c5
      [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1714309299215
      [main] INFO org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer - [Consumer clientId=1-1, groupId=1] Subscribed to topic(s): test-topic1
      [consumer_background_thread] INFO org.apache.kafka.clients.Metadata - [Consumer clientId=1-1, groupId=1] Cluster ID: Some(MkU3OEVBNTcwNTJENDM2Qk)

      Stuck In here...

       

      Broker logs 

      broker    | [2024-04-28 12:42:27,751] INFO Sent auto-creation request for Set(__consumer_offsets) to the active controller. (kafka.server.DefaultAutoTopicCreationManager)
      broker    | [2024-04-28 12:42:27,801] INFO Sent auto-creation request for Set(__consumer_offsets) to the active controller. (kafka.server.DefaultAutoTopicCreationManager)
      broker    | [2024-04-28 12:42:28,211] INFO Sent auto-creation request for Set(__consumer_offsets) to the active controller. (kafka.server.DefaultAutoTopicCreationManager)
      broker    | [2024-04-28 12:42:28,259] INFO Sent auto-creation request for Set(__consumer_offsets) to the active controller. (kafka.server.DefaultAutoTopicCreationManager)
      broker    | [2024-04-28 12:42:28,727] INFO Sent auto-creation request for Set(__consumer_offsets) to the active controller. (kafka.server.DefaultAutoTopicCreationManager)

      stuck in here

      Attachments

        1. image-2024-04-30-08-33-50-435.png
          107 kB
          sanghyeok An
        2. image-2024-04-30-08-33-06-367.png
          87 kB
          sanghyeok An

        Issue Links

          Activity

            People

              kirktrue Kirk True
              chickenchickenlove sanghyeok An
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: