Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10746

Consumer poll timeout Expiration should be logged as WARNING not INFO.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.5.0, 2.6.0, 2.5.1
    • 3.0.0
    • clients
    • None

    Description

      When a consumer does not poll regularly, and the `max.poll.interval.ms` threshold is reached, the consumer leaves the consumer group, and the reason is logged as an INFO message:

      https://github.com/a0x8o/kafka/blob/e032a4ad9bac2392b4406d0a3b245d6011edd15b/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L1356

      https://github.com/a0x8o/kafka/blob/e032a4ad9bac2392b4406d0a3b245d6011edd15b/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L1016

      Most Kafka users ignore INFO messages or have the log level set to WARN. Still many users run into this issue, since their applications take too long to process the polled records, and then the consumer fails to commit the offsets, which leads to duplicate message processing. Not seeing the error message in the first place means that users lose a lot of time debugging and searching for the reason for duplicate message processing.

      Therefore it seems like the log level of this message should be increased to WARN. 

      Attachments

        Issue Links

          Activity

            People

              showuon Luke Chen
              benedikt Benedikt Linse
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: