I had not arrived at trying the integration tests before (sorry), I'm looking into it now.
I identified the following problems:
- whirr.instance-templates: the thriftserver was run on a node on which there is no hadoop. This does not work as then we can't copy over the hadoop-core jar. This will also be a problem for
WHIRR-339 (hbase-site.xml generation), I'll add a comment over there. The 'solution' is to adjust the template, I think this is fair enough since there are other limitations too on how templates must be structured.
- Whirr decides to first configure the node with the regionserver, and after that the one with zookeeper. This fails because the regionserver will only wait/retry a limited amount of time to connect to ZooKeeper on startup (20s, I checked in the logs that zookeeper actually started after the regionserver exited, about 10s later). Luckily, there is a property to control this: hbase.zookeeper.recoverable.waittime. I put it to 5 minutes (this prop only has effect on startup AFAICS).
With these changes, the test runs successfully for me.
To make the tests work for both 0.89 and 0.90, I had to add an additional test class and properties file, the name of the properties file is passed to HBaseServiceController.getInstance(). Let me know if you prefer to have it done some other way.
Note that, in services/hbase/pom.xml, I left the hbase.version property on 0.89etcetera which does work for both since it uses the thrift interface which seems to be compatible.
The tests for 0.89 still run, but since copying hadoop-core is done regardless of the HBase version, the above mentioned limitation of needing to have Hadoop installed on the node with thrift now also applies to 0.89.
The hbase.zookeeper.recoverable.waittime is specified in the cluster configuration, which depends on
WHIRR-339. Without WHIRR-339, it needs to be hardcoded in the configure_(cdh_)hbase.sh
Through the properties passed with -Dconfig to the integration test, I supplied the following:
whirr.image-id=eu-west-1/ami-619ea915 (canonical 11.04 instance store EU)