As a process of getting to this target, one of the activities was to identify a subset of tests that would run in close to 10 minutes and would have a reasonable coverage. The idea is to have the main core flow/ core components well tested by the fast test suite and the corner cases/library classes could afford to be left out. Needless to say, the entire test suite will be a part of the nightly test cycle.
Attaching a spread sheet as a first step in this identification/classification of test suites into Fast Tests. The run time of these tests is marginally over 10 minutes (The entire test suite runs in about 2 hours). Effort is still on to improve a few more test cases that would hopefully bring this run time down even further.
The attached file has three sheets –
- Proposed set of tests that could make up the Fast Test suite,
- Analysis of existing tests in the mapred package and whether they have been considered for fast tests or not, with some notes
- List of classes where the coverage drop between the fast tests and all tests is greater than 10%. This also has information of where the drop is coming from
Clover was used for measuring code coverage.
Please share your thoughts/suggestions/ideas