Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6320

Flakey JobManagerHAJobGraphRecoveryITCase

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.3.1, 1.4.0
    • Component/s: Tests
    • Labels:

      Description

      it looks as if there is a race condition in the cleanup of JobManagerHAJobGraphRecoveryITCase.

      Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 50.271 sec <<< FAILURE! - in org.apache.flink.test.recovery.JobManagerHAJobGraphRecoveryITCase
      testJobPersistencyWhenJobManagerShutdown(org.apache.flink.test.recovery.JobManagerHAJobGraphRecoveryITCase)  Time elapsed: 0.129 sec  <<< ERROR!
      java.io.FileNotFoundException: File does not exist: /tmp/9b63934b-789d-428c-aa9e-47d5d8fa1e32/recovery/submittedJobGraphf763d61fba47
      	at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2275)
      	at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653)
      	at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535)
      	at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270)
      	at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653)
      	at org.apache.flink.test.recovery.JobManagerHAJobGraphRecoveryITCase.cleanUp(JobManagerHAJobGraphRecoveryITCase.java:112)
      

      Full log: https://s3.amazonaws.com/archive.travis-ci.org/jobs/223124016/log.txt

      Maybe a rule-based temporary directory is a better solution:

      	@Rule
      	public TemporaryFolder tempFolder = new TemporaryFolder();
      

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user NicoK opened a pull request:

          https://github.com/apache/flink/pull/3966

          FLINK-6320 fix unit test failing sometimes when deleting a temp dir

          use a JUnit rule with a `TemporaryFolder` instead which should fix this (alternatively, catching the exception may have fixed this symptom but using a temporary folder should be even more save in case a real concurrency issue was present)

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/NicoK/flink flink-6320

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3966.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3966


          commit 5b440c4084076c8086d888c4d5439c2e5f65bdf6
          Author: Nico Kruber <nico@data-artisans.com>
          Date: 2017-05-22T14:31:08Z

          FLINK-6320 fix unit test failing sometimes when deleting a temp directory


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3966 FLINK-6320 fix unit test failing sometimes when deleting a temp dir use a JUnit rule with a `TemporaryFolder` instead which should fix this (alternatively, catching the exception may have fixed this symptom but using a temporary folder should be even more save in case a real concurrency issue was present) You can merge this pull request into a Git repository by running: $ git pull https://github.com/NicoK/flink flink-6320 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3966.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3966 commit 5b440c4084076c8086d888c4d5439c2e5f65bdf6 Author: Nico Kruber <nico@data-artisans.com> Date: 2017-05-22T14:31:08Z FLINK-6320 fix unit test failing sometimes when deleting a temp directory
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/3966

          +1 to merge.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/3966 +1 to merge.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/3966

          merging.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/3966 merging.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3966

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3966
          Hide
          Zentol Chesnay Schepler added a comment -

          1.4: 6ad3d140f35722055c9011dbee88d19319cfbfbe

          Show
          Zentol Chesnay Schepler added a comment - 1.4: 6ad3d140f35722055c9011dbee88d19319cfbfbe
          Hide
          Zentol Chesnay Schepler added a comment -

          I'll also push this one to 1.3.

          Show
          Zentol Chesnay Schepler added a comment - I'll also push this one to 1.3.
          Hide
          Zentol Chesnay Schepler added a comment -

          1.3: d1ae1cd73665c6870111a1d1d413e41dd77da095

          Marked as fixed for 1.3.1 as I'm not sure if it will be included in 1.3.

          Show
          Zentol Chesnay Schepler added a comment - 1.3: d1ae1cd73665c6870111a1d1d413e41dd77da095 Marked as fixed for 1.3.1 as I'm not sure if it will be included in 1.3.

            People

            • Assignee:
              NicoK Nico Kruber
              Reporter:
              NicoK Nico Kruber
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development