[FLUME-2799] Kafka Source - Message Offset and Partition add to headers

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Implemented
• Affects Version/s: 1.6.0
• Fix Version/s: 1.7.0
• Component/s: Sinks+Sources
• Labels: easyfix patch
• Flags: Patch
• Assignee: Unassigned
• Reporter: Michael Andre Pearce
• Created: 21/Sep/15 12:56
• Updated: 22/Nov/18 16:54
• Resolved: 20/Jun/16 18:29

Description

Currently Kafka source only persists the original kafka message's topic into the Flume event headers.

For downstream interceptors and sinks that may want to have available to them the partition and the offset , we need to add these.

Also it is noted that the conversion from MessageAndMetaData to FlumeEvent is not configurable unlike other sources such as JMS.

Issue Links

• is related to: FLUME-3081 KafkaSource outputs offset in header
• links to: GitHub Pull Request #238