[SPARK-21142] spark-streaming-kafka-0-10 has too fat dependency on kafka

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.1
• Fix Version/s: 2.3.0
• Component/s: DStreams
• Assignee: Tim Van Wassenhove
• Reporter: Tim Van Wassenhove
• Created: 19/Jun/17 14:47
• Updated: 20/Jul/17 17:21
• Resolved: 20/Jul/17 17:21

Description

Currently spark-streaming-kafka-0-10_2.11 has a dependency on kafka, where it should only need a dependency on kafka-clients.

The only reason there is a dependency on kafka (full server) is due to KafkaTestUtils class to run in memory tests against a kafka broker. This class should be moved to src/test.

Issue Links

• links to: [Github] Pull Request #18353 (timvw)