Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4745

get_existing_cluster() doesn't work with additional security groups

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.1.0
    • 1.1.2, 1.2.1
    • EC2
    • None

    Description

      The spark-ec2 script has a flag that allows you to add additional security groups to clusters when you launch. However, the get_existing_cluster() function cycles through active instances and only returns instances whose group_names == cluster_name + "-master" (or + "-slaves"), which is the group created by default. The get_existing_cluster() function is used to login to, stop, and destroy existing clusters, among other actions.

      This is a pretty simple fix for which I've already submitted a pull request. It checks if cluster_name + "-master" is in the list of groups for each active instance. This means the cluster group can be one among many groups, rather than the sole group for an instance.

      Attachments

        Activity

          People

            alexdebrie Alex DeBrie
            alexdebrie Alex DeBrie
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: