Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-993

Offset Management API is either broken or mis-documented

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0, 0.8.1
    • Fix Version/s: None
    • Component/s: network
    • Labels:
      None

      Description

      I am in the process of building a set of Go client bindings for the new 0.8 protocol (https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol). Everything works but the Offset Commit/Fetch APIs. Fetch never returns any data, and trying to Commit results in the broker forcibly disconnecting my client. I have double-checked the bytes on the wire using Wireshark, and my client is obeying the protocol spec.

      After some digging, I found KAFKA-852 which seems related, but I have tried my client against the 0.8 beta, 0.8 branch, and even trunk with the same results.

      When I try and commit, the stack-trace that the broker produces is:
      [2013-07-31 10:34:14,423] ERROR Closing socket for /192.168.12.71 because of error (kafka.network.Processor)
      java.nio.BufferUnderflowException
      at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:127)
      at java.nio.ByteBuffer.get(ByteBuffer.java:675)
      at kafka.api.ApiUtils$.readShortString(ApiUtils.scala:38)
      at kafka.api.UpdateMetadataRequest$$anonfun$readFrom$1.apply(UpdateMetadataRequest.scala:42)
      at kafka.api.UpdateMetadataRequest$$anonfun$readFrom$1.apply(UpdateMetadataRequest.scala:41)
      at scala.collection.immutable.Range$ByOne$class.foreach(Range.scala:282)
      at scala.collection.immutable.Range$$anon$2.foreach(Range.scala:265)
      at kafka.api.UpdateMetadataRequest$.readFrom(UpdateMetadataRequest.scala:41)
      at kafka.api.RequestKeys$$anonfun$7.apply(RequestKeys.scala:42)
      at kafka.api.RequestKeys$$anonfun$7.apply(RequestKeys.scala:42)
      at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:49)
      at kafka.network.Processor.read(SocketServer.scala:345)
      at kafka.network.Processor.run(SocketServer.scala:245)
      at java.lang.Thread.run(Thread.java:680)

      Is this a bug, or is the protocol spec wrong? Also, since I can't seem to find a straight answer anywhere else: is offset fetch/commit expected to be in 0.8, 0.8.1, or some later release?

      Thanks,
      Evan

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                junrao Jun Rao
                Reporter:
                eapache Evan Huus
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: