[SPARK-5629] Add spark-ec2 action to return info about an existing cluster

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Component/s: EC2
• Assignee: Unassigned
• Reporter: Nicholas Chammas
• Created: 05/Feb/15 20:22
• Updated: 09/Feb/16 09:13
• Resolved: 09/Feb/16 09:13

Description

You can launch multiple clusters using spark-ec2. At some point, you might just want to get some information about an existing cluster.

Use cases include:

• Wanting to check something about your cluster in the EC2 web console.
• Wanting to feed information about your cluster to another tool (e.g. as described in SPARK-5627).

So, in addition to the existing actions:

• launch
• destroy
• login
• stop
• start
• get-master
• reboot-slaves

We add a new action, describe, which describes an existing cluster if given a cluster name, and all clusters if not.

Some examples:

# describes all clusters launched by spark-ec2
spark-ec2 describe

# describes cluster-1
spark-ec2 describe cluster-1

In combination with the proposal in SPARK-5627:

# describes cluster-3 in a machine-readable way (e.g. JSON)
spark-ec2 describe cluster-3 --machine-readable

Parallels in similar tools include:

• juju status from Ubuntu Juju
• starcluster listclusters from MIT StarCluster

Issue Links

• is duplicated by: SPARK-5879 spary_ec2.py should expose/return master and slave lists (e.g. write to file)
• is related to: SPARK-5627 Enhance spark-ec2 to return machine-readable output