Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2781

A Kafka Channel defined as parseAsFlumeEvent=false cannot be correctly used by a Flume source

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.0
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Labels:
    • Release Note:
      When Flume writes to a channel defined as parseAsFlumeEvent=false, use text instead of Avro

      Description

      When a Kafka channel is configured as parseAsFlumeEvent=false, the channel will read events from the topic as text instead of serialized Avro Flume events.
      This is useful so Flume can read from an existing Kafka topic, where other Kafka clients publish as text.

      However, if you use a Flume source on that channel, it will still write the events as Avro so it will create an inconsistency and those events will fail to be read correctly.

      Also, this would allow a Flume source to write to a Kafka channel and any Kafka subscriber to listen to Flume events passing through without binary dependencies.

        Attachments

          Activity

            People

            • Assignee:
              gherreros Gonzalo Herreros
              Reporter:
              gherreros Gonzalo Herreros

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment