Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16234

Fix unstable cases in StreamingJobGraphGeneratorTest

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Hide
      The specification about HashSet says that "it makes no guarantees as to the iteration order of the set; in particular, it does not guarantee that the order will remain constant over time".
      The documentation here is for your reference:
      https://docs.oracle.com/javase/8/docs/api/java/util/HashSet.html
      Show
      The specification about HashSet says that "it makes no guarantees as to the iteration order of the set; in particular, it does not guarantee that the order will remain constant over time". The documentation here is for your reference: https://docs.oracle.com/javase/8/docs/api/java/util/HashSet.html

    Description

      The test in org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest#testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled will cause the following failure:

      java.lang.AssertionError: expected:<SlotSharingGroup feca28aff5a3958840bee985ee7de4d3> but was:<SlotSharingGroup 798f7268aeb5fde00858b7c9723d65f1>
      at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.assertSameSlotSharingGroup(StreamingJobGraphGeneratorTest.java:843)
      at org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:814)

      I analyze the assertion failure and find that the root cause of it lies in the clear() method in StreamGraph.java, where the variable sources is initialized as a HashSet. Because the iteration order of HashSet is non-deterministic, so the test becomes flaky. 

      The fix is to change HashSet to LinkedHashSet and then the failure above is removed. 

       

      The stacktrace information is presented as follows for your reference:

      java.util.HashSet.iterator(HashSet.java:173)
      org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.setChaining(StreamingJobGraphGenerator.java:251)
      org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:166)
      org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:104)
      org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:100)
      org.apache.flink.streaming.api.graph.StreamingJobGraphGeneratorTest.testSlotSharingOnAllVerticesInSameSlotSharingGroupByDefaultDisabled(StreamingJobGraphGeneratorTest.java:803)

       

       

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            cpugputpu cpugputpu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m

                Slack

                  Issue deployment