[WICKET-7088] Improve test reliability by resolving nondeterministic order of Set and Map

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 10.0.0-M2, 9.16.0
• Fix Version/s: 10.0.0
• Component/s: wicket-core
• Labels: pull-request-available

Environment

Reproduce
This error can be reproduced with the NondexTool with the following command (you can try that without modifying the pom, but feel free to use the plugin to protect your project and make it safer 😊 ), and this kind problem is widely documented in International Dataset of Flaky Tests (IDoFT)

mvn edu.illinois:nondex-maven-plugin:2.1.7-SNAPSHOT:nondex -pl ./wicket-core-tests -DnondexRuns=10 -Dtest=org.apache.wicket.markup.html.form.CollectionFormComponentTest#getSetUnmodifiableSet
Most similar Similar issues

FasterXML/jackson-jaxrs-providers#154
https://github.com/FasterXML/jackson-annotations/pull/194/files

• Assignee: Martin Tzvetanov Grigorov
• Reporter: huiyang chi
• Created: 27/Nov/23 15:30
• Updated: 04/Dec/23 16:01
• Resolved: 04/Dec/23 07:36

Description

The related PR had been approved  : https://github.com/apache/wicket/pull/707

Problem

In the test getSetUnmodifiableSet and getSetNullList, we use a set to intitialize our choice, and assume it will have the order with getDefaultModelObjectAsString to be [A, B], but this is not necessary true according to Oracle's official document

 

For the test renderAjaxAttributes, we create the Json string from the AjaxRequestAttributes and assume the order of the fields in tests which is not true because it serialize from extraParameters which was a HashMap, which did not guarantee any order and will cause non-deterministic when we change the environments like JDK version. Also the LinkedHashMap is designed with specification to guarantee the order to resolve this kind of problem

Solution

Use LinkedHashMap

This problem had encountered and solved in wicket the same way before :
wicket/wicket-core/src/main/java/org/apache/wicket/ajax/json/README

Lines 4 to 6 in b243eca

IMPORTANT: JSONObject#map field is manually changed to LinkedHashMap (from HashMap) to keep
the order of the ajax attributes consistent for WicketTester based unit tests

 

 

Affected Tests

 

[INFO] org.apache.wicket.markup.html.form.CollectionFormComponentTest#getSetUnmodifiableSet
[INFO] org.apache.wicket.markup.html.form.CollectionFormComponentTest#getSetNullSet

[INFO] org.apache.wicket.ajax.AbstractDefaultAjaxBehaviorTest#renderAjaxAttributes
[INFO] org.apache.wicket.ajax.AjaxCallbackFunctionTest#testCallbackFunctionWithConverted
[INFO] org.apache.wicket.ajax.AjaxCallbackFunctionTest#testCallbackFunctionWithExplicit
[INFO] org.apache.wicket.ajax.AjaxCallbackFunctionTest#testCallbackFunctionWithAll
[INFO] org.apache.wicket.ajax.AjaxCallbackFunctionTest#testCallbackFunctionWithContext
[INFO] org.apache.wicket.ajax.AjaxCallbackFunctionTest#testJQueryUIEvent
[INFO] org.apache.wicket.ajax.AjaxCallbackFunctionTest#testCallbackFunctionWithResolved
[INFO] org.apache.wicket.ajax.AjaxCallbackFunctionTest#testDefaultCallbackFunction

Issue Links

• links to: GitHub Pull Request #707