Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2495

Kafka Source may miss events when channel is not available

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.6.0
    • Sinks+Sources
    • None

    Description

      Because the Kafka consumer itself tracks offsets, and we don't restart the consumer when we get channel errors, the consumer will skip messages that we couldn't write to channel, even though we did not advance offset in Zookeeper.

      Attachments

        1. FLUME-2495.0.patch
          6 kB
          Gwen Shapira
        2. FLUME-2495.1.patch
          7 kB
          Gwen Shapira
        3. FLUME-2495.2.patch
          8 kB
          Gwen Shapira

        Activity

          People

            gwenshap Gwen Shapira
            gwenshap Gwen Shapira
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: