Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16235

Use LinkedHashSet in StreamGraph.java

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Docs Text:
      Hide
      The specification about HashSet says that "it makes no guarantees as to the iteration order of the set; in particular, it does not guarantee that the order will remain constant over time".
      The documentation here is for your reference:
      https://docs.oracle.com/javase/8/docs/api/java/util/HashSet.html
      Show
      The specification about HashSet says that "it makes no guarantees as to the iteration order of the set; in particular, it does not guarantee that the order will remain constant over time". The documentation here is for your reference: https://docs.oracle.com/javase/8/docs/api/java/util/HashSet.html

      Description

      The test in org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest#testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled will cause the following failure:

      java.lang.AssertionError: expected:<SlotSharingGroup feca28aff5a3958840bee985ee7de4d3> but was:<SlotSharingGroup 798f7268aeb5fde00858b7c9723d65f1>
      at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.assertSameSlotSharingGroup(StreamingJobGraphGeneratorTest.java:843)
      at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:814)

      I analyze the assertion failure and find that the root cause of it lies in the clear() method in StreamGraph.java, where the variable sources is initialized as a HashSet. Because the iteration order of HashSet is non-deterministic, so the test becomes flaky. 

      The fix is to change HashSet to LinkedHashSet and then the failure above is removed. 

       

      The stacktrace information is presented as follows for your reference:

      java.util.HashSet.iterator(HashSet.java:173)
      org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.setChaining(StreamingJobGraphGenerator.java:251)
      org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:166)
      org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:104)
      org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:100)
      org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:803)

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                cpugputpu cpugputpu
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: