Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2495

Kafka Source may miss events when channel is not available

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      Because the Kafka consumer itself tracks offsets, and we don't restart the consumer when we get channel errors, the consumer will skip messages that we couldn't write to channel, even though we did not advance offset in Zookeeper.

        Attachments

        1. FLUME-2495.2.patch
          8 kB
          Gwen Shapira
        2. FLUME-2495.1.patch
          7 kB
          Gwen Shapira
        3. FLUME-2495.0.patch
          6 kB
          Gwen Shapira

          Activity

            People

            • Assignee:
              gwenshap Gwen Shapira
              Reporter:
              gwenshap Gwen Shapira
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: