[SPARK-7038] [Streaming] Spark Sink requires spark assembly in classpath

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 1.3.1
• Component/s: DStreams
• Labels: bulk-closed
• Target Version/s: 1.3.2
• Assignee: Hari Shreedharan
• Reporter: Hari Shreedharan
• Shepherd: Hari Shreedharan
• Created: 21/Apr/15 19:27
• Updated: 21/May/19 04:32
• Resolved: 21/May/19 04:32

Description

In 1.3.0 Spark, we shaded Guava, which meant that the Spark Sink's guava dependency is not standard guava anymore - thus the one from Flume's classpath does not work and can throw a NoClassDefFoundError while using Spark Sink.

We must pull in the guava dependency into the Spark Sink jar.