Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6380

BlobService concurrency issues between delete and put/get methods

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.3.0
    • 1.4.0
    • Runtime / Network
    • None

    Description

      BlobCache#deleteAll(JobID) deletes the job directory which is only created at the start of BlobCache#getURL(BlobKey) which then relies on the directory being present.

      This is not restricted to the BlobCache, though, but also affects the BlobServer in two ways:
      1) its own local storage and
      2) its backing BlobStore

      For the latter, i.e. in FileSystemBlobStore, there is no guarantee that a directory will not be deleted concurrently (from a delete method) between its creation and writing a file (in a get method):

      • the delete method for name-addressable blobs always deletes the job-specific storage directory if there is no further blob for this job
      • the content-addressable blobs do that similarly but are shared among jobs and thus only delete directories if there is no other blob.

      Since name-addressable blobs have not been used so far and the latter case typically does not occur concurrently with get/put requests, this has not been a problem so far but is more relevant after applying FLINK-6046.

      Attachments

        Issue Links

          Activity

            People

              nkruber Nico Kruber
              nkruber Nico Kruber
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: