Affects Version/s: 1.1.1, 1.2.0
Fix Version/s: None
- Spark 1.1.1 or 1.2.0
- EC2 cluster with 1 slave spun up using spark-ec2
- twitter4j 3.0.3
- spark-shell called with --jars argument to load spark-streaming-twitter_2.10-1.0.0.jar as well as all the twitter4j jars.
(Continuing the discussion started here on the Spark user list.)
The following Spark Streaming code throws a serialization exception I do not understand.
So before I even start the StreamingContext, I get the following stack trace:
What I'm really trying to do is use Spark Streaming via the interactive shell to filter Tweets using a trained KMeans model. I got errors trying that, and I boiled it down to this repro.