Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-3081

KafkaSource outputs offset in header

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.7.0, 1.8.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Adding message offset to KafkaSource headers.
    • Flags:
      Patch

      Description

      In FLUME-2799 the original request included (amongst others) Kafka offset to be part of the message headers from the KafkaSource. However the ticket was resolved without it. In affected versions the possible headers are:

      • topic
      • partition
      • key
      • timestamp (agent JVM system time)

      I propose to add offset in order to make a traceable system key (topic+partition+offset) in order to synchronise between several consumers (something necessary for my case).

        Attachments

        1. FLUME-3081-0.patch
          2 kB
          Viktor Gerdin

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                viktor.gerdin Viktor Gerdin
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: