[FLUME-2789] Kafka sink sends messages to the source topic by mistake

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.6.0
• Component/s: Sinks+Sources

Environment

Flume 1.6

• Assignee: Unassigned
• Reporter: Alex Tian
• Created: 09/Sep/15 15:06
• Updated: 19/May/16 17:06

Description

In my scenario, I need to send messages from a kafka source to a kafka sink , in other workds, transfering messages from a topic A to another topic B.
As a result, I find my kafka sink always sends messages back to topic A.

The reason in the codes is:
KafkaSink.java:
****************************************************************************************
107 if ((eventTopic = headers.get(TOPIC_HDR)) == null)

{ 108 eventTopic = topic; 109 }

****************************************************************************************

If the source is a Kafka source, eventTopic won't be null because it is the source topic from the msg header. Therefore, the kafka sink will send the message to its source topic by mistake.

Issue Links

• is duplicated by: FLUME-2814 flume kafka sink does not write events to configured sink topic when source is also from other topic of kafka
• is related to: FLUME-2848 Kafka Source could not work with Kafka Sink