Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1961

HBase EC2 scripts

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Minor
    • Resolution: Not A Problem
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Amazon AWS EC2

      Description

      Attached tarball is a clone of the Hadoop EC2 scripts, modified significantly to start up a HBase storage only cluster on top of HDFS backed by instance storage.

      Tested with the HBase 0.20 branch but should work with trunk also. Only the AMI create and launch scripts are tested. Will bring up a functioning HBase cluster.

      Do "create-hbase-image c1.xlarge" to create an x86_64 AMI, or "create-hbase-image c1.medium" to create an i386 AMI. Public Hadoop/HBase 0.20.1 AMIs are available:
      i386: ami-c644a7af
      x86_64: ami-f244a79b

      launch-hbase-cluster brings up the cluster: First, a small dedicated ZK quorum, specifiable in size, default of 3. Then, the DFS namenode (formatting on first boot) and one datanode and the HBase master. Then, a specifiable number of slaves, instances running DFS datanodes and HBase region servers. For example:

          launch-hbase-cluster testcluster 100 5
      

      would bring up a cluster with 100 slaves supported by a 5 node ZK ensemble.

      We must colocate a datanode with the namenode because currently the master won't tolerate a brand new DFS with only namenode and no datanodes up yet. See HBASE-1960. By default the launch scripts provision ZooKeeper as c1.medium and the HBase master and region servers as c1.xlarge. The result is a HBase cluster supported by a ZooKeeper ensemble. ZK ensembles are not dynamic, but HBase clusters can be grown by simply starting up more slaves, just like Hadoop.

      hbase-ec2-init-remote.sh can be trivially edited to bring up a jobtracker on the master node and task trackers on the slaves.

        Attachments

        There are no Sub-Tasks for this issue.

          Activity

            People

            • Assignee:
              apurtell Andrew Kyle Purtell
              Reporter:
              apurtell Andrew Kyle Purtell
            • Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: