Kafka
  1. Kafka
  2. KAFKA-242

Subsequent calls of ConsumerConnector.createMessageStreams cause Consumer offset to be incorrect

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.7
    • Fix Version/s: None
    • Component/s: clients
    • Labels:
      None

      Description

      When calling ConsumerConnector.createMessageStreams in rapid succession, the Consumer offset is incorrectly advanced causing the consumer to lose messages. This seems to happen when createMessageStreams is called before the rebalancing triggered by the previous call to createMessageStreams has completed.

      1. kafka.log
        7 kB
        David Arthur

        Activity

        Hide
        David Arthur added a comment -

        Annotated log messages from two different test runs. For each test run, 20k messages were written to "mytopic" and "mytopic1" each with no consumers running. Once the messages had all been sent, the consumers were started up

        Show
        David Arthur added a comment - Annotated log messages from two different test runs. For each test run, 20k messages were written to "mytopic" and "mytopic1" each with no consumers running. Once the messages had all been sent, the consumers were started up
        Hide
        Jay Kreps added a comment -

        Does this bug still exist in 0.8?

        Show
        Jay Kreps added a comment - Does this bug still exist in 0.8?

          People

          • Assignee:
            Unassigned
            Reporter:
            David Arthur
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development