Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10746

Consumer poll timeout Expiration should be logged as WARNING not INFO.

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.5.0, 2.6.0, 2.5.1
    • Fix Version/s: 3.0.0
    • Component/s: clients
    • Labels:
      None

      Description

      When a consumer does not poll regularly, and the `max.poll.interval.ms` threshold is reached, the consumer leaves the consumer group, and the reason is logged as an INFO message:

      https://github.com/a0x8o/kafka/blob/e032a4ad9bac2392b4406d0a3b245d6011edd15b/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L1356

      https://github.com/a0x8o/kafka/blob/e032a4ad9bac2392b4406d0a3b245d6011edd15b/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L1016

      Most Kafka users ignore INFO messages or have the log level set to WARN. Still many users run into this issue, since their applications take too long to process the polled records, and then the consumer fails to commit the offsets, which leads to duplicate message processing. Not seeing the error message in the first place means that users lose a lot of time debugging and searching for the reason for duplicate message processing.

      Therefore it seems like the log level of this message should be increased to WARN. 

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                showuon Luke Chen
                Reporter:
                benedikt Benedikt Linse
              • Votes:
                1 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: