Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6450

Kafka Transaction with auto-commit=false, showing transaction log as a consumer lag

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.0.0
    • None
    • consumer
    • None
    • Mac and Linux

    Description

      We posted a message to topic with transaction. For testing purposes we are using single message. Offset incremented by 2, 1 for message and 1 for transaction
       
      D25SP18BGQ17IM:confluent-4.0.0 mxp8051$ ./bin/kafka-run-class kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic transaction-test
      transaction-test:0:6
       
      Pushed another message to same topic with transaction. Offset incremented by 2.
       
      D25SP18BGQ17IM:confluent-4.0.0 mxp8051$ ./bin/kafka-run-class kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic transaction-test
      transaction-test:0:8
       
      Consumer lag shows 4 when my consumer is down.
       
      D25SP18BGQ17IM:confluent-4.0.0 mxp8051$ ./bin/kafka-consumer-groups --bootstrap-server localhost:9092  --describe  --group transaction
      Note: This will not show information about old Zookeeper-based consumers.
       
      Consumer group 'transaction' has no active members.
       
      TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG        CONSUMER-ID                                       HOST                           CLIENT-ID
      transaction-test               0          4               8               4          -    
       
      Started the consumer and lag says 1
       
      D25SP18BGQ17IM:confluent-4.0.0 mxp8051$ ./bin/kafka-consumer-groups --bootstrap-server localhost:9092  --describe  --group transaction
      Note: This will not show information about old Zookeeper-based consumers.
       
      TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG        CONSUMER-ID                                       HOST                           CLIENT-ID
      transaction-test               0          7               8               1          consumer-1-7a792d01-f3df-4e33-9ccc-636f8110ba38   /172.22.52.231                 consumer-1

      Please see google group discussion for the same at [#https://groups.google.com/d/msg/confluent-platform/IQKd3BKgvYw/hkS1hgiYAQAJ]

      Attachments

        Activity

          People

            Unassigned Unassigned
            mayurpatil Mayur Patil
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: