Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4942

Kafka Connect: Offset committing times out before expected

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.10.2.0
    • 0.11.0.0
    • connect
    • None

    Description

      On Kafka 0.10.2.0
      I run a connector that deals with a lot of data, in a kafka connect cluster

      When the offsets are getting committed, I get the following:

      [2017-03-23 03:56:25,134] INFO WorkerSinkTask{id=MyConnector-1} Committing offsets (org.apache.kafka.connect.runtime.WorkerSinkTask)
      [2017-03-23 03:56:25,135] WARN Commit of WorkerSinkTask{id=MyConnector-1} offsets timed out (org.apache.kafka.connect.runtime.WorkerSinkTask)
      

      If you look at the timestamps, they're 1 ms apart. My settings are the following:

      	offset.flush.interval.ms = 120000
      	offset.flush.timeout.ms = 60000
      	offset.storage.topic = _connect_offsets
      

      It seems the offset flush timeout setting is completely ignored for the look of the logs. I would expect the timeout message to happen 60 seconds after the commit offset INFO message, not 1 millisecond later.

      Attachments

        Activity

          People

            Unassigned Unassigned
            stephane.maarek@gmail.com Stephane Maarek
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: