> Can you please describe the test plan you used to actually test end-to-end coverage of rackspace?
I tested this by manually spinning up a cluster, running a job, checking the results and terminating the cluster. The commands to do this are in the README.
I've now modified the integration test script to run on Rackspace, which I successfully ran with the following (note that the image ID is private, since Rackspace does not support shared images):
HADOOP_CLOUD_PROVIDER=rackspace IMAGE_ID=200181 INSTANCE_TYPE=4 HADOOP_HOME=~/dev/hadoop-0.20.1/ ./transient-cluster.sh
> regarding the patch:
> see comments re. TERREMARK_KEY in
HADOOP-6465 for RACKSPACE_KEY, etc.
Agree that this would be a useful way to specify keys. I've created
HADOOP-6480 for this.
> re. boot script: I noticed a lot of very similar code in the terremark boot script. Can some of this be factored out into a common utility script that gets sourced in? This seems to be a looming maintenence problem.
This could become a problem, but I'd like to address it in a separate issue.
> RackspaceCluster.wait_for_instances() looks like a clone of TerremarkCluster.wait_for_instances(). Can this be pushed up into cluster?
> Ditto with _all_started(), _ip_list_to_string(), terminate()?
These are the same methods. I'd like to commit this one first, then I'll re-work
HADOOP-6465 so the methods are pulled up into Cluster.
> Why is rackspace remote-setup.sh strictly-ubuntu-based? The boot script looks like it also handles centos.
I've only tested on Ubuntu so far, but it would be straightforward to run on Centos too. I'll create another JIRA for this.
Thanks for the review!