[BIGTOP-1222] Simplify and gradleize a subset of the bigtop smokes

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.7.0
• Fix Version/s: 0.8.0
• Component/s: build, tests
• Assignee: Konstantin I Boudnik
• Reporter: jay vyas
• Created: 25/Feb/14 22:06
• Updated: 27/Mar/15 13:44
• Resolved: 20/Aug/14 04:03

Description

(Rewritten the description for clarity)

We need an easier way to run bigtop smoke tests, and gradle provides this:

1) Easy to script/modify
2) Human readable
3) equally oriented towards both groovy and plain old java

The advantage of this method to running smokes :

1) No need to compile a jar : this is a costly step and not much value added, also creates indirection which can make debugging a broken test very hard.

2) Simple: A smoke test doesnt need to make low level API calls or be compiled against the right APIs - rather, it should test the end user interface ("hive -q ....", "pig -x ....", "hadoop jar ....", and so on).

3) Customizable: The smoke tests shouldnt require users to have to write XML and debug environmental variables / grep around for System properties etc. Rather, a high level controller should do all that checking for you.

The initial idea was to write a python/bash implementation wrapper of scripts, but that was replaced by the idea of using gradle. The advantage of gradle is that we don't need to manually set the classpath and run groovy commands: Gradle wraps groovy scripts in their native java context quite nicely - but it doesnt add any other unnecessary overhead (xml, jar files, no need for complex xml tag wrappers for simple tasks - just plain groovy code).

So, here the goal is just to create a nice, clean, extensible non-jar, non-API dependent gradle runner for the smoke tests which exersizes the hadoop cluster the same way a typical end-user would.

Issue Links

• blocks: BIGTOP-1333 Update build.gradle in smoketests to read from paramterized file input
• blocks: BIGTOP-1315 Pig smoke tests: Refactor ?
• blocks: BIGTOP-1384 Implement Gradle Wrapper for smoke tests and cleanup.
• breaks: BIGTOP-1663 TestHadoopSmoke is failing because of improper set-up
• contains: BIGTOP-1315 Pig smoke tests: Refactor ?
• contains: BIGTOP-1019 Remove mysql requirement constraint from sqoop tests
• incorporates: BIGTOP-1372 Bigtop needs feature that takes in multiple arguments to build specific components at command line
• is blocked by: BIGTOP-1402 Update bigtop gradle to 2.x
• is blocked by: BIGTOP-1405 Gradle 2.0 installation from toolchain is broken
• is related to: BIGTOP-1305 Use tests from the HBase test jar to potentially replace hbase-smokes
• is related to: BIGTOP-1195 Higher level interface to smokes.
• is required by: BIGTOP-1309 Gradle environment overhaul
• relates to: BIGTOP-847 form VM baking come up with a way to init services that require being up and running
• relates to: BIGTOP-685 provide a way to specify the parameters expected by a test

Sub-Tasks

• Update build.gradle in smoketests to read from paramterized file input: Open