Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5630

Consumer poll loop over the same record after a CorruptRecordException

    XMLWordPrintableJSON

Details

    Description

      Hello

      While consuming a topic with log compaction enabled, I am getting an infinite consumption loop of the same record, i.e, each call to poll is returning to me 500 times one record (500 is my max.poll.records). I am using the java client 0.11.0.0.

      Running the code with the debugger, the initial problem come from `Fetcher.PartitionRecords,fetchRecords()`.
      Here I get a `org.apache.kafka.common.errors.CorruptRecordException: Record size is less than the minimum record overhead (14)`
      Then the boolean `hasExceptionInLastFetch` is set to true, resulting the test block in `Fetcher.PartitionRecords.nextFetchedRecord()` to always return the last record.

      I guess the corruption problem is similar too https://issues.apache.org/jira/browse/KAFKA-5582 but this behavior of the client is probably not the expected one

      Attachments

        Issue Links

          Activity

            People

              becket_qin Jiangjie Qin
              vmaurin_glispa Vincent Maurin
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: