Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2781

A Kafka Channel defined as parseAsFlumeEvent=false cannot be correctly used by a Flume source

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.0
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Labels:
    • Release Note:
      When Flume writes to a channel defined as parseAsFlumeEvent=false, use text instead of Avro

      Description

      When a Kafka channel is configured as parseAsFlumeEvent=false, the channel will read events from the topic as text instead of serialized Avro Flume events.
      This is useful so Flume can read from an existing Kafka topic, where other Kafka clients publish as text.

      However, if you use a Flume source on that channel, it will still write the events as Avro so it will create an inconsistency and those events will fail to be read correctly.

      Also, this would allow a Flume source to write to a Kafka channel and any Kafka subscriber to listen to Flume events passing through without binary dependencies.

        Attachments

        1. FLUME-2781.patch
          4 kB
          Gonzalo Herreros

          Activity

            People

            • Assignee:
              gherreros Gonzalo Herreros
              Reporter:
              gherreros Gonzalo Herreros
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: