[SPARK-13877] Consider removing Kafka modules from Spark / Spark Streaming

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.6.1
• Component/s: DStreams, Spark Core
• Assignee: Unassigned
• Reporter: Hari Shreedharan
• Created: 14/Mar/16 22:55
• Updated: 03/Apr/16 00:55
• Resolved: 03/Apr/16 00:55

Description

Based on the discussion the PR for SPARK-13843 (here), we should consider moving the Kafka modules out of Spark as well.

Providing newer functionality (like security) has become painful while maintaining compatibility with older versions of Kafka. Moving this out allows more flexibility, allowing users to mix and match Kafka and Spark versions.

Issue Links

• relates to: SPARK-13843 Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages