Bug 24106 - Suites are not reported correctly in JUnitReport
Summary: Suites are not reported correctly in JUnitReport
Status: NEW
Alias: None
Product: Ant
Classification: Unclassified
Component: Optional Tasks (show other bugs)
Version: 1.6.1
Hardware: All Windows XP
: P3 normal with 8 votes (vote)
Target Milestone: ---
Assignee: Ant Notifications List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-10-24 15:43 UTC by Rusty Putzler
Modified: 2012-12-06 18:13 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Rusty Putzler 2003-10-24 15:43:46 UTC
I have a TestCase that does something like this:

public void MyTest extends TestCase(){
Comment 1 Rusty Putzler 2003-10-24 15:56:58 UTC
I have a TestCase that does something like this:

public void MyTest extends TestCase(){
	public static Test suite("MyTestSuite") {
		TestSuite suite = new TestSuite();
		suite.addTest(new MyTest("testPerformance"));
		suite.addTest(new MyTest("testPerformance2"));
		suite.addTest(new CustomDataPoolTest("testPerformance"));
		suite.addTest(new CustomDataPoolTest("privatePerformanceTest"));
		return suite;
	}
	public void testPerformance(){
		//do something
	}
	public void testPerformance2(){
		//do something
	}
  ....
}

The output of the Junit Report will record:

Mytest
	testPerformance
	testPerformance2
	testPerformance
	privatePerformance



There doesn't appear to be a way to determine that the second testPerformance 
or privatePerformance tests orginated from a different TestCase class. It seems 
like the report should list the suite name instead of the TestCase Name.  For 
instance:


MyTestSuite
	MyTest.testPerformance
	MyTest.testPerformance2
	CustomDataPoolTest.testPerformance
	CustomDataPoolTest.privatePerformance
Comment 2 Jeff 2004-06-29 17:02:19 UTC
Actually the XML formatter of the JUnit task outputs in this way.  JUnitReport
merely reformats the xml that was generated.  So it would appear that the XML
formatter should be rewritten or a new formatter added.  This is still a problem
in 1.6.1.  Has anyone looked into writing a new formatter?
Comment 3 Antoine Levy-Lambert 2006-10-28 17:11:30 UTC
This issue is very popular. Any one wants to write a patch for it ?
Comment 4 Dan Fabulich 2008-02-15 18:09:47 UTC
FYI, I spent some time working on this today, not really because I need it, but
because people keep complaining to me that Maven Surefire is broken because it's
incompatible with JUnitReport.  :-)

http://www.nabble.com/Test-Suites,-Ant,-Surefire,-and-JunitReport-td15076378.html

Ant 1.6 (I think?) introduced a helpful attribute on all of its <testcase>
classes to include the class name of the class on every <testcase> element:

     <testsuite name="com.mycompany.MyTestSuite" failures="0" errors="0"
tests="4" time="0.031">
       <properties><!-- ... --></properties
       <testcase name="test1a" time="0.0" classname="com.mycompany.Test1" />
       <testcase name="test1b" time="0.0" classname="com.mycompany.Test1" />
       <testcase name="test2a" time="0.0" classname="com.mycompany.Test2" />
       <testcase name="test2b" time="0.0" classname="com.mycompany.Test2" />
       <testcase name="test3a" time="0.0" classname="com.mycompany.Test3" />
       <testcase name="test3b" time="0.0" classname="com.mycompany.Test3" />
     </testsuite>

This is good, because it allows you to figure out which class the method really
belongs to.  This is what both Surefire and TestNG output when they generate
JUnit-like results.  Unfortunately, nobody honors the "classname" attribute,
including Ant 1.7's JunitReport task!

Further on in the thread linked above, Steve suggested I try my hand at updating
the XSLT.  I assert that fixing the XSLT without changing Ant is/would be
fricking impossible.  We could maybe handle it if the XML were organized like this:

     <testsuite name="com.mycompany.MyTestSuite" failures="0" errors="0"
tests="4" time="0.031">
       <properties><!-- ... --></properties>
       <package name="com.mycompany">
         <class name="Test1" />
           <testcase name="test1a" time="0.0"/>
           <testcase name="test1b" time="0.0"/>
         </class>
         <class name="Test2" />
           <testcase name="test2a" time="0.0"/>
           <testcase name="test2b" time="0.0"/>
         </class>
         <class name="Test3" />
           <testcase name="test3a" time="0.0"/>
           <testcase name="test3b" time="0.0"/>
         </class>
       </package>
     </testsuite>

But getting from the first schema into the second schema is brutally challenging
in XSLT 1.0 (the version with which we'd have to remain compatible).  

The hardest part is trying to reorganize the test cases into classes, without
the use of XSLT 1.1 node-sets.  You've got to loop through the test cases,
recognize classes that you've already seen, and group them together when they match.

Even just splitting the class string into its package, while possible, is really
annoying in XSLT; right now Ant does it at the suite level in Java in
XMLResultAggregator.

If an XSLT masochist, er, I mean WIZARD wants to take a crack at this problem,
I'm sure it'd be appreciated.  junit-noframes.xsl is only a few hundred lines
long; how bad could it be? ;-)

I suppose I could put together a patch in XMLResultAggregator, but then it's not
entirely clear whether my goal should be to remain backwards compatible with the
existing XSLT that's out there (presumably by splitting <testsuite> elements up
into multiple <testsuite> elements in the TESTS-TestSuites.xml file) or what.
Comment 5 Jan Cumps 2008-02-16 02:15:48 UTC
I would prefer the 'evolving' approach as documented on wiki: 
Proposals/EnhancedTestReports: 
http://wiki.apache.org/ant/Proposals/EnhancedTestReports.

I think that what you try to achieve can be done by the current XML output 
format.
Sorting and grouping in the standard XSLT files has already been achieved for 
package.

Comment 6 Andrei Costescu 2012-12-06 16:57:44 UTC
See 48529. I did a patch for this, but it didn't get any attention right away. Don't know how hard it is to merge that to the latest version.
Comment 7 Jesse Glick 2012-12-06 18:13:48 UTC
Duplicate of bug #48529?