Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7487

DumpLogSegments reports mismatches for indexed offsets which are not at the start of a record batch

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.3.0
    • Component/s: core
    • Labels:

      Description

      When running DumpLogSegments against an .index file, mismatches may be reported when the indexed message offset is not the first record in a batch. For example:

       Mismatches in :/var/lib/kafka/data/replicated-topic-0/00000000000000000000.index
       Index offset: 968, log offset: 966
      

      And looking at the corresponding .log file:

      baseOffset: 966 lastOffset: 968 count: 3 baseSequence: -1 lastSequence: -1 producerId: -1 producerEpoch: -1 partitionLeaderEpoch: 0 isTransactional: false position: 3952771 CreateTime: 1538768639065 isvalid: true size: 12166 magic: 2 compresscodec: NONE crc: 294402254 
      

      In this case, the last offset in the batch was indexed instead of the first, but the index has to map physical position to the start of the batch, leading to the mismatch.

      It seems like DumpLogSegments should not report these cases as mismatches, which users might interpret as an error or problem

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ijuma Ismael Juma
                Reporter:
                mikebin Michael Bingham
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: