Whirr
  1. Whirr
  2. WHIRR-461

Allow user to specify spot instance price per instance template group

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None

      Description

      A natural improvement on top of WHIRR-332 is to allow the user to specify the spot instance price per instance template group.

      Proposed syntax:

      whirr.templates.role1+role2.aws-ec2-spot-price=0.23
      

      Using this feature we can have a Hadoop cluster that has both regular and spot instance like this:

      whirr.cluster-name=hadoop-on-ec2
      whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 3 hadoop-datanode+hadoop-jobtracker, 7 hadoop-datanode+hadoop-jobtracker+noop
      
      # user a large instance for the namenode as needed
      whirr.templates.hadoop-namenode+hadoop-jobtracker.hardware-id=c1.xlarge
      
      # regular smaller instance for a few data nodes
      whirr.templates.hadoop-datanode+hadoop-jobtracker.hardware-id=m1.large
      
      # and spot instances for the rest of the datanodes
      whirr.templats.hadoop-datanode+hadoop-tasktracker+noop.aws-ec2-spot-price=0.32
      
      
      1. WHIRR-461.patch
        23 kB
        Andrei Savu

        Activity

        Hide
        Andrei Savu added a comment -

        Patch changes:

        • cleanup InstanceTemplate + support for spot price
        • unit tests

        I have done some manual testing on aws-ec2 and everything seems to be working fine.

        Show
        Andrei Savu added a comment - Patch changes: cleanup InstanceTemplate + support for spot price unit tests I have done some manual testing on aws-ec2 and everything seems to be working fine.
        Hide
        Andrei Savu added a comment -

        Also tested with ZooKeeper: 2 regular + 1 spot.

        Show
        Andrei Savu added a comment - Also tested with ZooKeeper: 2 regular + 1 spot.
        Hide
        Adrian Cole added a comment -

        looks good, except I'd prefer clearer scoping on custom properties like spot price to use the provider id:

        instead of: aws-ec2-spot-price, aws-ec2.spot-price

        Show
        Adrian Cole added a comment - looks good, except I'd prefer clearer scoping on custom properties like spot price to use the provider id: instead of: aws-ec2-spot-price, aws-ec2.spot-price
        Hide
        Andrei Savu added a comment -

        I'm using aws-ec2-spot-price for consistency with whirr.aws-ec2-spot-price.

        Show
        Andrei Savu added a comment - I'm using aws-ec2-spot-price for consistency with whirr.aws-ec2-spot-price.
        Hide
        Adrian Cole added a comment -

        okie then +1 as my comment is more general than this change

        Show
        Adrian Cole added a comment - okie then +1 as my comment is more general than this change
        Hide
        Andrei Savu added a comment -

        Committed. Thanks Adrian for reviewing. We can address the naming issue in another JIRA.

        Show
        Andrei Savu added a comment - Committed. Thanks Adrian for reviewing. We can address the naming issue in another JIRA.

          People

          • Assignee:
            Andrei Savu
            Reporter:
            Andrei Savu
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development