Kafka
  1. Kafka
  2. KAFKA-748

Append to index fails due to invalid offset

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:

      Description

      We seem to be appending to the index and it checks to make sure we do not insert an entry with an offset that is larger than the actual offset of the message. We seem to be trying to insert an offset = 1 in the index while lastOffset is 24463. This seems to get fixed on restarting the broker.

      java.lang.IllegalArgumentException: Attempt to append an offset (1) to position 21703 no larger than the last offset appended (24463).
      at kafka.log.OffsetIndex.append(OffsetIndex.scala:183)
      at kafka.log.LogSegment.append(LogSegment.scala:60)
      at kafka.log.Log.append(Log.scala:286)
      at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:188)
      at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:181)
      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
      at scala.collection.immutable.Map$Map1.foreach(Map.scala:105)
      at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)

      1. KAFKA-748-v1.patch
        6 kB
        Jay Kreps
      2. outmsg
        3.00 MB
        Sriram Subramanian
      3. outindex
        699 kB
        Sriram Subramanian

        Activity

        Neha Narkhede made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Jay Kreps made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Jay Kreps made changes -
        Attachment KAFKA-748-v1.patch [ 12568498 ]
        Sriram Subramanian made changes -
        Assignee Sriram Subramanian [ sriramsub ] Jay Kreps [ jkreps ]
        Sriram Subramanian made changes -
        Attachment outindex [ 12567905 ]
        Attachment outmsg [ 12567906 ]
        Neha Narkhede made changes -
        Labels p1
        Neha Narkhede made changes -
        Field Original Value New Value
        Priority Major [ 3 ] Blocker [ 1 ]
        Sriram Subramanian created issue -

          People

          • Assignee:
            Jay Kreps
            Reporter:
            Sriram Subramanian
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development