Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5630

Consumer poll loop over the same record after a CorruptRecordException

    XMLWordPrintableJSON

    Details

      Description

      Hello

      While consuming a topic with log compaction enabled, I am getting an infinite consumption loop of the same record, i.e, each call to poll is returning to me 500 times one record (500 is my max.poll.records). I am using the java client 0.11.0.0.

      Running the code with the debugger, the initial problem come from `Fetcher.PartitionRecords,fetchRecords()`.
      Here I get a `org.apache.kafka.common.errors.CorruptRecordException: Record size is less than the minimum record overhead (14)`
      Then the boolean `hasExceptionInLastFetch` is set to true, resulting the test block in `Fetcher.PartitionRecords.nextFetchedRecord()` to always return the last record.

      I guess the corruption problem is similar too https://issues.apache.org/jira/browse/KAFKA-5582 but this behavior of the client is probably not the expected one

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                becket_qin Jiangjie Qin
                Reporter:
                vmaurin_glispa Vincent Maurin
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: