Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4745

get_existing_cluster() doesn't work with additional security groups

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.1.2, 1.2.1
    • Component/s: EC2
    • Labels:
      None

      Description

      The spark-ec2 script has a flag that allows you to add additional security groups to clusters when you launch. However, the get_existing_cluster() function cycles through active instances and only returns instances whose group_names == cluster_name + "-master" (or + "-slaves"), which is the group created by default. The get_existing_cluster() function is used to login to, stop, and destroy existing clusters, among other actions.

      This is a pretty simple fix for which I've already submitted a pull request. It checks if cluster_name + "-master" is in the list of groups for each active instance. This means the cluster group can be one among many groups, rather than the sole group for an instance.

        Attachments

          Activity

            People

            • Assignee:
              alexdebrie Alex DeBrie
              Reporter:
              alexdebrie Alex DeBrie
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: