[FLUME-3281] Update to Kafka 2.0 client

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.0
• Fix Version/s: 1.9.0
• Assignee: Endre Major
• Reporter: Endre Major
• Due: 23/Nov/18
• Created: 07/Nov/18 15:50
• Updated: 14/Feb/20 07:50
• Resolved: 21/Nov/18 10:24

Description

Kafka client 2.0 has been out for a while. It would be nice to use a more recent client.
Flume uses the Kafka client and the server libraries. Server libraries are required for offset migration. Offset migration should be marked deprecated so Flume 2.0 could get rid of the Kafka server dependencies.

Issue Links

• is related to: FLUME-3051 Mention the incompatibility of Kafka source with 0.8.x Kafka brokers
• relates to: BIGTOP-3305 Bump Flume to 1.9.0
• relates to: FLUME-3252 kafka upgrade
• relates to: FLUME-3197 Upgrade Kafka client to 1.0.x
• links to: GitHub Pull Request #235