[FLUME-2855] Add kafka 0.9 support for Flume

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.7.0
• Component/s: Channel, Sinks+Sources
• Release Note: Add support for kafka 0.9.0.0 clients API
• Assignee: Unassigned
• Reporter: Grigoriy Rozhkov
• Created: 14/Dec/15 11:59
• Updated: 25/Jun/16 06:25
• Resolved: 25/Jun/16 06:25

Description

Kafka 0.9 already released and it introduces new clients API that not compatible with old one.
Add support for Apache Flume to use new kafka consumer and producer API. Old classes will not be removed to save backward compatibility with old Kafka Source, Sink and Channel implementations. User will not need to change agent configuration if wants to use old Kafka. Kafka API version will able to be configured in agent configuration file.

1. Add new Kafka Source implementation which uses new Kafka consumer API.
2. Add new Kafka Sink implementation which uses new Kafka producer API.
3. Add new Kafka Channel implementation.

Issue Links

• is related to: FLUME-2821 Flume-Kafka Source with new Consumer
• links to: review board