Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-8148

Test instability in YarnFileStageTest

    XMLWordPrintableJSON

    Details

      Description

      Running org.apache.flink.yarn.YarnFileStageTestS3ITCase
      Tests run: 3, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 13.152 sec <<< FAILURE! - in org.apache.flink.yarn.YarnFileStageTestS3ITCase
      testRecursiveUploadForYarnS3(org.apache.flink.yarn.YarnFileStageTestS3ITCase)  Time elapsed: 8.515 sec  <<< FAILURE!
      java.lang.AssertionError: null
      	at org.junit.Assert.fail(Assert.java:86)
      	at org.junit.Assert.assertTrue(Assert.java:41)
      	at org.junit.Assert.assertFalse(Assert.java:64)
      	at org.junit.Assert.assertFalse(Assert.java:74)
      	at org.apache.flink.yarn.YarnFileStageTestS3ITCase.testRecursiveUploadForYarn(YarnFileStageTestS3ITCase.java:171)
      	at org.apache.flink.yarn.YarnFileStageTestS3ITCase.testRecursiveUploadForYarnS3(YarnFileStageTestS3ITCase.java:192)
      

      from https://travis-ci.org/apache/flink/jobs/305861539

      YarnFileStageTestS3ITCase.testRecursiveUploadForYarn verifies that the test directory used is cleaned up by YarnFileStageTest.testCopyFromLocalRecursive which should clean up the directory (in a finally block). However, for S3, we may not always see our own deletes.

      Quoting from https://aws.amazon.com/s3/faqs/ here:

      Q: What data consistency model does Amazon S3 employ?
      Amazon S3 buckets in all Regions provide read-after-write consistency for PUTS of new objects and eventual consistency for overwrite PUTS and DELETES.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                NicoK Nico Kruber
                Reporter:
                NicoK Nico Kruber
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: