Bug 52323 - Cobertura test code coverage support for build.xml and build.properties.default
Cobertura test code coverage support for build.xml and build.properties.default
Status: RESOLVED FIXED
Product: Tomcat 7
Classification: Unclassified
Component: Packaging
trunk
PC Linux
: P2 enhancement (vote)
: ---
Assigned To: Tomcat Developers Mailing List
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2011-12-13 13:57 UTC by mhasko
Modified: 2013-11-05 10:17 UTC (History)
0 users



Attachments
the patch, based on revision 1213685 (5.73 KB, patch)
2011-12-13 13:57 UTC, mhasko
Details | Diff
updated patch (6.60 KB, patch)
2013-11-02 12:51 UTC, Violeta Georgieva
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description mhasko 2011-12-13 13:57:56 UTC
Created attachment 28069 [details]
the patch, based on revision 1213685

Cobertura (cobertura.sf.net) is an opensource Java code coverage tool released (with in dependencies) under The Apache Software License, Version 1.1 and/or The GNU General Public License, Version 2.0.

It can assess how much of the codebase was actually executed during a program run.
If the program run means running the test suite, it will produce results showing how much of the code the tests cover, potentially discover weak spots.

This patch provides support for Cobertura in the build.xml and build.properties.default files. It introduces four new targets:
 * download-cobertura - downloads and unpacks Cobertura in ${base.path}
 * cobertura-enable - enables Cobertura to be used
 * cobertura-instrument - adds Cobertura instrumentation to compiled bytecode
 * cobertura-report - produces reports from Cobertura results. HTML by default
   (can be overwritten with 'cobertura.report.format' property).

The behavior of this patch doesn't break existing targets in any way.

To use this tool, one simply runs 'ant cobertura-enable test' which will instrument the compiled classes before running the tests and also supply all the necessary resources for Cobertura to gather the coverage results.
Once the tests are completed 'ant cobertura-report' will produce lovely HTML report about the test coverage in '${tomcat.build}/coverage'.

NOTE: Although the result of the test cases should not be affected by Cobertura, it is not guaranteed.
Comment 1 Mark Thomas 2011-12-13 19:36:05 UTC
Marking as an enhancement
Comment 2 Mark Thomas 2013-11-01 10:09:01 UTC
Just tried to enable Cobertura with Tomcat 8 trunk and the tests fail consistently with NPE's in the instrumented code. Absent a working patch for this with Tomncat 8, this issue is eventually going to be resolved as WONTFIX.
Comment 3 Violeta Georgieva 2013-11-02 12:51:59 UTC
Created attachment 30996 [details]
updated patch

Hi,

I touched the script here and there and also updated the version to 2.0.3.
I think it is working now. Can you check? I tested on Windows.

The NPE was because the cobertura instrumented our ClassLoaderLogManager and in the same time tried to initialize a static Logger field. [1]

Regards
Violeta

[1] https://github.com/cobertura/cobertura/blob/master/cobertura/src/main/java/net/sourceforge/cobertura/coveragedata/TouchCollector.java
Comment 4 Violeta Georgieva 2013-11-02 12:57:12 UTC
The patch is against trunk (Tomcat 8)
Comment 5 Mark Thomas 2013-11-04 09:20:06 UTC
Ah, that makes sense.

I changed the way the patch worked to make it consistent with how we enable things like validation.

Cobertura support has been added to 8.0.x (8.0.0-RC6 onwards) and 7.0.x (7.0.48 onwards).
Comment 6 Christopher Schultz 2013-11-04 15:06:24 UTC
(In reply to Mark Thomas from comment #5)
> Ah, that makes sense.
> 
> I changed the way the patch worked to make it consistent with how we enable
> things like validation.
> 
> Cobertura support has been added to 8.0.x (8.0.0-RC6 onwards) and 7.0.x
> (7.0.48 onwards).

So... what does our code-coverage look like? Is there a CC-report available through Jenkins?
Comment 7 Konstantin Kolinko 2013-11-04 20:57:14 UTC
1. It needs some documentation in BUILDING.txt
E.g. at "(7.3) Other configuration options".


Testing current trunk, with JDK 7u45. Ant 1.9.2
Cobertura is NOT enabled.
Running only NIO tests.
execute.test.nio=true

ant test 1>ant_test.log 2>&1

I see two problems:


2. Cobertura is downloaded even though I have not enabled it.

Apparently the "download-cobertura" target needs if="${test.cobertura}".


3. I see a lot of error messages in the test logs, though the tests themselves do not fail.

E.g. in 
TEST-org.apache.catalina.core.TestStandardContextResources.NIO.txt

05-Nov-2013 00:35:52.555 SEVERE [localhost-startStop-1] org.apache.catalina.startup.ContextConfig.processAnnotationsJar Unable to process Jar entry [com/ibm/icu/impl/data/LocaleElements_zh__PINYIN.class] from Jar [jar:file:/(...)/libraries/tomcat8/cobertura-2.0.3/lib/icu4j-2.6.1.jar!/] for annotations
 org.apache.tomcat.util.bcel.classfile.ClassFormatException: Invalid byte tag in constant pool: 60
	at org.apache.tomcat.util.bcel.classfile.Constant.readConstant(Constant.java:124)
	at org.apache.tomcat.util.bcel.classfile.ConstantPool.<init>(ConstantPool.java:60)
	at org.apache.tomcat.util.bcel.classfile.ClassParser.readConstantPool(ClassParser.java:167)
	at org.apache.tomcat.util.bcel.classfile.ClassParser.parse(ClassParser.java:91)
	at org.apache.catalina.startup.ContextConfig.processAnnotationsStream(ContextConfig.java:2027)
	at org.apache.catalina.startup.ContextConfig.processAnnotationsJar(ContextConfig.java:1958)
	at org.apache.catalina.startup.ContextConfig.processAnnotationsUrl(ContextConfig.java:1926)
	at org.apache.catalina.startup.ContextConfig.processAnnotations(ContextConfig.java:1877)
	at org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1147)
	at org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:769)
	at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:305)
	at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
	at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5166)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1390)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1380)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
Comment 8 Konstantin Kolinko 2013-11-04 23:58:08 UTC
(In reply to Konstantin Kolinko from comment #7)
> 2. Cobertura is downloaded even though I have not enabled it.
> 
> Apparently the "download-cobertura" target needs if="${test.cobertura}".
> 

"2." has been fixed in trunk and Tomcat 7.
Comment 9 Mark Thomas 2013-11-05 10:13:57 UTC
(In reply to Christopher Schultz from comment #6)
> 
> So... what does our code-coverage look like? Is there a CC-report available
> through Jenkins?

Tomcat doesn't use the ASF's Jenkins server for CI, it uses Buildbot.

I've just enabled Cobertura for the trunk build. If I have got it right, something should appear here in the next few hours:
http://ci.apache.org/projects/tomcat/tomcat8/coverage
Comment 10 Mark Thomas 2013-11-05 10:17:34 UTC
1. & 3. have also been fixed in trunk and 7.0.x