[SPARK-9494] 'spark-ec2 launch' fails with anaconda python 3.4

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 1.4.1
• Fix Version/s: 1.6.0
• Component/s: EC2

Environment

OSX, Anaconda, Python 3.4

• Assignee: Unassigned
• Reporter: Stuart Owen
• Created: 30/Jul/15 23:44
• Updated: 30/Jan/16 00:43
• Resolved: 30/Jan/16 00:43

Description

Command I used to launch:

$SPARK_HOME/ec2/spark-ec2 \
    -k spark \
    -i ~/keys/spark.pem \
    -s $NUM_SLAVES \
    --copy-aws-credentials \
    --region=us-east-1 \
    --instance-type=m3.2xlarge \
    --spot-price=0.1 \
    launch $CLUSTER_NAME

Execution log:

/Users/stuart/Applications/anaconda/lib/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  PendingDeprecationWarning)
Setting up security groups...
Searching for existing cluster july-event-fix in region us-east-1...
Spark AMI: ami-35b1885c
Launching instances...
Traceback (most recent call last):
  File "/Users/stuart/Applications/spark-1.4.0-bin-hadoop2.6/ec2/spark_ec2.py", line 1455, in <module>
    main()
  File "/Users/stuart/Applications/spark-1.4.0-bin-hadoop2.6/ec2/spark_ec2.py", line 1447, in main
    real_main()
  File "/Users/stuart/Applications/spark-1.4.0-bin-hadoop2.6/ec2/spark_ec2.py", line 1276, in real_main
    (master_nodes, slave_nodes) = launch_cluster(conn, opts, cluster_name)
  File "/Users/stuart/Applications/spark-1.4.0-bin-hadoop2.6/ec2/spark_ec2.py", line 566, in launch_cluster
    name = '/dev/sd' + string.letters[i + 1]
AttributeError: 'module' object has no attribute 'letters'
/Users/stuart/Applications/anaconda/lib/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  PendingDeprecationWarning)
ERROR: Could not find a master for cluster july-event-fix in region us-east-1.
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.1.2', 55678), raddr=('207.171.162.181', 443)>

Issue Links

• duplicates: SPARK-11837 spark_ec2.py breaks with python3 and m3 instances