Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14831 Über-jira: S3a phase IV: Hadoop 3.1 features
  3. HADOOP-13974

S3Guard CLI to support list/purge of pending multipart commits

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-beta1
    • 3.1.0
    • fs/s3
    • None

    Description

      The S3A CLI will need to be able to list and delete pending multipart commits.

      We can do the cleanup already via fs.s3a properties. The CLI will let scripts stat for outstanding data (have a different exit code) and permit batch jobs to explicitly trigger cleanups.

      This will become critical with the multipart committer, as there's a significantly higher likelihood of commits remaining outstanding.

      We may also want to be able to enumerate/cancel all pending commits in the FS tree

      Attachments

        1. HADOOP-13974.007.patch
          57 kB
          Aaron Fabbri
        2. HADOOP-13974.006.patch
          57 kB
          Aaron Fabbri
        3. HADOOP-13974.005.patch
          54 kB
          Aaron Fabbri
        4. HADOOP-13974.004.patch
          46 kB
          Aaron Fabbri
        5. HADOOP-13974.003.patch
          43 kB
          Aaron Fabbri
        6. HADOOP-13974.002.patch
          41 kB
          Aaron Fabbri
        7. HADOOP-13974.001.patch
          37 kB
          Aaron Fabbri

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            fabbri Aaron Fabbri
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment