Description
(apologies if I already had a ticket for this somewhere, I couldn't find it)
Our integration tests are very nice and automated at the moment because we can use MiniAccumuloCluster to "provision" an Accumulo instance (or used a shared instance), and run a test against it. For the most part, this works well and provides an accurate test harness.
Thus, to run integration tests, you need a sufficiently beefy machine since the same host will be running all of Accumulo as well as performing any client work. When resources are available to use, it would be nice to leverage them – whether these are yarn, mesos, a vanila installation, etc.
In addition to the additional computational power from using extra hardware, it also encourages us to use the public API as much as possible instead of relying on "hidden" impl methods in MiniAccumuloCluster.
I propose making changes to the IT test base (AbstractMacIT, SimpleMacIT, ConfigurableMacIT) to add an extra step between them an test classes to allow "injection" of a more generic Accumulo "cluster" that is not associated with MAC.
Attachments
Issue Links
- relates to
-
ACCUMULO-3366 Ensure ITs can run against cluster with SSL
- Resolved
-
ACCUMULO-3367 Lift alter-and-reset-configuration paradigm in common location
- Resolved
- links to
1.
|
Remove internal use of deprecated test classes | Resolved | Christopher Tubbs |
|
My initial thoughts on how to implement this is to introduce an extra shim between integration tests and the underlying cluster implementation (e.g. SimpleMacIT). Thus, our ITs that need a running instance of Accumulo will extend this new class that provides a "deployment agnostic" set of functions to interact with that instance.
Some complexity issues arise around this because some deployments (notably, the plain-jane Accumulo instance) cannot guarantee the breadth of functionality that some tests expect: the inability to restart Accumulo processes, control site configuration files (HDFS or Accumulo), and alter JVM level properties. All other configuration should be capable of modification via use of ZooKeeper through the normal API. MAC and Slider (Accumulo on YARN) both solve these problems because they can dynamically start a new instance with the requested configuration – I've been referring to this as "managed" clusters.
My initial intent is to support running tests against MAC as currently happens (the default won't change), provide some configuration to allow use an existing vanilla Accumulo instance, ability to skip tests which require "managed" clusters, and, if not full support initially, changes which will help Slider integration down the road.