Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-10213

Camel Kafka doesn't support setting "max.poll.records" and "session.timeout.ms"

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.18.0
    • 2.18.0
    • camel-kafka
    • None
    • Novice

    Description

      Currently it is not possible to control the processing time in the poll loop in Camel Kafka consumers.

      It is important to have the possibility to control the poll loop by adjusting max.poll.records and session.timeout.ms . Otherwise this warning is produced when consuming a large backlog from kafka topic:

      Auto offset commit failed for group consumerGroup: Commit cannot be 
      completed since the group has already rebalanced and assigned the 
      partitions to another member. This means that the time between 
      subsequent calls to poll() was longer than the configured 
      session.timeout.ms, which typically implies that the poll loop is spending 
      too much time message processing. You can address this either by 
      increasing the session timeout or by reducing the maximum size of batches
      returned in poll() with max.poll.records.
      

      Please add max.poll.records and session.timeout.ms to Kafka endpoint configuration.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            davsclaus Claus Ibsen
            OHolzma Oliver Holzmann
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment