[HBASE-2327] [EC2] Allocate elastic IP addresses for ZK and master nodes

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.90.0
• Assignee: Andrew Kyle Purtell
• Reporter: Andrew Kyle Purtell
• Created: 15/Mar/10 22:08
• Updated: 20/Nov/15 12:43
• Resolved: 16/Mar/10 00:20

Description

Amazon EC2 supports Elastic IP Addresses to implement the effect of having a static IP address for public servers running on EC2. Up on hbase-users@ there was some recent discussion, confirmed, that when an EC2 instance queries the external DNS name of an elastic IP address, EC2 DNS returns the internal IP address of the instance to which the elastic IP address is bound, so it is safe to use elastic IPs for the ZK and master nodes. We gain the ability to do transparent replacement of one instance, e.g. failed, with another without incurring any additional cost.

Update launch-hbase-zookeeper and launch-hbase-master to allocate elastic IPs:

  $ ec2-allocate-address 
  ADDRESS 1.1.1.1

and then assign the elastic IP address to the appropriate instance(s):

$ ec2-associate-address -i i-11111111 1.1.1.1
ADDRESS 1.1.1.1  i-11111111

and then get the external DNS name to use when performing substitutions on master and slave configs:

$ ec2-describe-instances i-11111111 | egrep ^INSTANCE | cut -f4
ec2-1-1-1-1.compute-1.amazonaws.com

When shutting down the cluster, just release the elastic IPs after terminating the instances:

ec2-release-address 1.1.1.1
...

NOTE: AWS accounts default to a limit of 5 Elastic IP addresses but most will run with 1 master and 3 or 1 ZK instances. And, the ZK ensemble can be shared.

A follow up issue can address making scripts to launch replacements for failed instances transparently.