[SPARK-14648] Spark EC2 script creates cluster but spark is not installed properly.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.1
• Component/s: EC2

Environment

Spark 1.6.1

• Assignee: Josh Rosen
• Reporter: Nikhil
• Created: 15/Apr/16 01:14
• Updated: 19/May/16 15:09
• Resolved: 17/Apr/16 00:06

Description

Launched EC2 cluster using this code

./spark-ec2 --key-pair=spark --identity-file=spark.pem --region=us-east-1 --instance-type=c3.large -s 1 --copy-aws-credentials launch test-cluster

Spark cluster created and running.

After login, spark folder does not contain anything.
-------------------------------------------------------------------

When the script ran, here's where the problem was I believe.

Saving to: ‘spark-1.6.1-bin-hadoop1.tgz’

100%[====================================================================================================================================================================================================>] 277,258,240 55.8MB/s in 4.4s

2016-04-14 23:31:49 (59.8 MB/s) - ‘spark-1.6.1-bin-hadoop1.tgz’ saved [277258240/277258240]

Unpacking Spark

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
mv: missing destination file operand after `spark'
Try `mv --help' for more information.
[timing] spark init: 00h 00m 06s