[BIGTOP-2834] spark charm: refactor for restricted networks; lib cleanup

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.3.0
• Component/s: deployment
• Assignee: Kevin Monroe
• Reporter: Kevin Monroe
• Created: 06/Jul/17 23:46
• Updated: 07/Jul/17 21:49
• Resolved: 07/Jul/17 21:49

Description

The spark charm currently grabs Spark-Bench and sample data for the pagerank benchmark from s3. This is not good for a couple reasons:

1 - Spark-Bench hasn't been verified with Spark 2.1.0.
2 - Deployments might fail in an environment where the charm is not allowed to fetch things from s3.

#1 is an easy fix; just change the default behavior. SB is there if a user wants it, but we shouldn't install it by default.

#2 is trickier because we do need the sample data for our pagerank benchmark. Refactor this as a Juju Resource (vs s3 URL) so it comes from the charm store.

While we're here, simplify the spark charm library to remove leftover cruft from pre-Bigtop days and break out the complex configure method into more purpose-optimized methods.

Issue Links

• links to: GitHub Pull Request #246