[SPARK-1574] ec2/spark_ec2.py should provide option to control number of attempts for ssh operations

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Component/s: EC2
• Assignee: Nicholas Chammas
• Reporter: Art Peel
• Created: 22/Apr/14 21:21
• Updated: 13/Dec/14 22:19
• Resolved: 13/Dec/14 22:19

Description

EC instances are sometimes slow to start up. When this happens, generating the cluster ssh key or sending the generated cluster key to the slaves can fail due to an ssh timeout.

The script currently hard-codes the number of tries for ssh operations as 2.

For more flexibility, it should be possible to specify the number of tries with a command-line option, --num-ssh-tries, that defaults to 2 to keep the current behavior if not provided.

Issue Links

• relates to: SPARK-3398 Have spark-ec2 intelligently wait for specific cluster states