Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-7266

Don't attempt to delete parent directory on S3

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 1.3.1
    • 1.3.2, 1.4.0
    • None
    • None

    Description

      Currently, every attempted release of an S3 state object also checks if the "parent directory" is empty and then tries to delete it.

      Not only is that unnecessary on S3, but it is prohibitively expensive and for example causes S3 to throttle calls by the JobManager on checkpoint cleanup.

      The FileState must only attempt parent directory cleanup when operating against real file systems, not when operating against object stores.

      Attachments

        Issue Links

          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-7265 FileSystems should describe their kind and consistency level

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. FLINK-7587 Flink doesn't delete empty checkpoint file's dir from s3

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          supercedes

          Bug - A problem which impairs or prevents the functions of the product. FLINK-7587 Flink doesn't delete empty checkpoint file's dir from s3

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Activity

            People

              aljoscha Aljoscha Krettek
              sewen Stephan Ewen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: