Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-7266

Don't attempt to delete parent directory on S3

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.3.1
    • Fix Version/s: 1.4.0, 1.3.2
    • Component/s: Core
    • Labels:
      None

      Description

      Currently, every attempted release of an S3 state object also checks if the "parent directory" is empty and then tries to delete it.

      Not only is that unnecessary on S3, but it is prohibitively expensive and for example causes S3 to throttle calls by the JobManager on checkpoint cleanup.

      The FileState must only attempt parent directory cleanup when operating against real file systems, not when operating against object stores.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                aljoscha Aljoscha Krettek
                Reporter:
                StephanEwen Stephan Ewen
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: