Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: HADOOP-13345
    • Fix Version/s: None
    • Component/s: fs/s3
    • Labels:
      None

      Description

      The S3A CLI will need to be able to list and delete pending multipart commits.

      We can do the cleanup already via fs.s3a properties. The CLI will let scripts stat for outstanding data (have a different exit code) and permit batch jobs to explicitly trigger cleanups.

      This will become critical with the multipart committer, as there's a significantly higher likelihood of commits remaining outstanding.

      We may also want to be able to enumerate/cancel all pending commits in the FS tree

        Activity

        Hide
        stevel@apache.org Steve Loughran added a comment -

        we can use List Multiparts to list active MPUs, something for tests to pick up as well (i.e. assert that there aren't any active)

        Show
        stevel@apache.org Steve Loughran added a comment - we can use List Multiparts to list active MPUs, something for tests to pick up as well (i.e. assert that there aren't any active)
        Hide
        Thomas Demoor Thomas Demoor added a comment -

        For purging we use TransferManagers abortMultipartUploads which under the hood does exactly that: listMultipartuploads and then iterate that list and abort 1 by 1.

        Show
        Thomas Demoor Thomas Demoor added a comment - For purging we use TransferManagers abortMultipartUploads which under the hood does exactly that: listMultipartuploads and then iterate that list and abort 1 by 1.

          People

          • Assignee:
            Unassigned
            Reporter:
            stevel@apache.org Steve Loughran
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development