Kafka
  1. Kafka
  2. KAFKA-748

Append to index fails due to invalid offset

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:

      Description

      We seem to be appending to the index and it checks to make sure we do not insert an entry with an offset that is larger than the actual offset of the message. We seem to be trying to insert an offset = 1 in the index while lastOffset is 24463. This seems to get fixed on restarting the broker.

      java.lang.IllegalArgumentException: Attempt to append an offset (1) to position 21703 no larger than the last offset appended (24463).
      at kafka.log.OffsetIndex.append(OffsetIndex.scala:183)
      at kafka.log.LogSegment.append(LogSegment.scala:60)
      at kafka.log.Log.append(Log.scala:286)
      at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:188)
      at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:181)
      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
      at scala.collection.immutable.Map$Map1.foreach(Map.scala:105)
      at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)

      1. outindex
        699 kB
        Sriram Subramanian
      2. outmsg
        3.00 MB
        Sriram Subramanian
      3. KAFKA-748-v1.patch
        6 kB
        Jay Kreps

        Activity

        Sriram Subramanian created issue -
        Neha Narkhede made changes -
        Field Original Value New Value
        Priority Major [ 3 ] Blocker [ 1 ]
        Neha Narkhede made changes -
        Labels p1
        Sriram Subramanian made changes -
        Attachment outindex [ 12567905 ]
        Attachment outmsg [ 12567906 ]
        Sriram Subramanian made changes -
        Assignee Sriram Subramanian [ sriramsub ] Jay Kreps [ jkreps ]
        Jay Kreps made changes -
        Attachment KAFKA-748-v1.patch [ 12568498 ]
        Jay Kreps made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Neha Narkhede made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Jay Kreps
            Reporter:
            Sriram Subramanian
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development