[FLINK-16234] Fix unstable cases in StreamingJobGraphGeneratorTest

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.10.1, 1.11.0
• Labels: pull-request-available

Docs Text

The specification about HashSet says that "it makes no guarantees as to the iteration order of the set; in particular, it does not guarantee that the order will remain constant over time".
The documentation here is for your reference:
https://docs.oracle.com/javase/8/docs/api/java/util/HashSet.html

• Assignee: Unassigned
• Reporter: cpugputpu
• Created: 22/Feb/20 14:50
• Updated: 27/Feb/20 07:39
• Resolved: 27/Feb/20 07:39

Description

The test in org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest#testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled will cause the following failure:

java.lang.AssertionError: expected:<SlotSharingGroup feca28aff5a3958840bee985ee7de4d3> but was:<SlotSharingGroup 798f7268aeb5fde00858b7c9723d65f1>
at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.assertSameSlotSharingGroup(StreamingJobGraphGeneratorTest.java:843)
at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:814)

I analyze the assertion failure and find that the root cause of it lies in the clear() method in StreamGraph.java, where the variable sources is initialized as a HashSet. Because the iteration order of HashSet is non-deterministic, so the test becomes flaky. 

The fix is to change HashSet to LinkedHashSet and then the failure above is removed. 

 

The stacktrace information is presented as follows for your reference:

java.util.HashSet.iterator(HashSet.java:173)
org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.setChaining(StreamingJobGraphGenerator.java:251)
org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:166)
org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:104)
org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:100)
org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:803)

 

 

Issue Links

• duplicates: FLINK-16235 Use LinkedHashSet in StreamGraph.java
• links to: GitHub Pull Request #11187