[SUREFIRE-1395] Surefire plugin reports bogus test failure

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 2.19, 2.20
• Component/s: Maven Surefire Plugin
• Environment: Windows 10 (1607) on x86_64 platform. JRE 1.8.0_141-b15 (tested with 1.8.0_131 too).
• Assignee: Tibor Digana
• Reporter: Carte Project
• Created: 24/Jul/17 20:31
• Updated: 15/Sep/17 16:12
• Resolved: 15/Sep/17 08:33

Description

Surefire plugin reports a bogus test failure when building and running the tests for an open source project. This is the report:

Tests run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 10.211 s <<< FAILURE! - in io.sf.carte.doc.html.DOMNodeTest
compareDocumentPosition1(io.sf.carte.doc.html.DOMNodeTest)  Time elapsed: 0.047 s  <<< FAILURE!
java.lang.AssertionError: expected:<10> but was:<10>
	at io.sf.carte.doc.html.DOMNodeTest.compareDocumentPosition1(DOMNodeTest.java:56)

The test is an integer comparison, and completes successfully when launched from Eclipse.

If anyone is interested in trying to reproduce it, do the following steps:

1) Download a dependency that is not in the central repository:
https://sourceforge.net/projects/jclf/files/latest/download
Run 'maven install' in the root directory of that project and install it on your maven local repository.

2) Download the code from https://sourceforge.net/projects/carte/files/css4j/0.36/css4j-0.36.zip/download and run 'mvn' at the directory. Now the error happens.

In principle my software setup is relatively simple, an up-to-date Windows 10-1607 with latest JDK8.

EDIT: more detailed steps to reproduce the problem.
EDIT 2: Updated link to file download.