Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18477 Über-jira: S3A Hadoop 3.3.9 features
  3. HADOOP-15575

Add a way for an FS instance to say "really, no trash interval at all"

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.0
    • None
    • fs, fs/s3
    • None

    Description

      Object stores like S3 often offer object versioning as a way to recover from accidental deletions. There is no need to use the Fs Shell trash mechanism.

      Proposed: a way for an FS instance to tell the shell to not use trash, even if the client has configured it. The current getServerDefaults().getTrashInterval() returns a value from the server, but it only overrides the ""fs.trash.interval" if it !=0. So if a server says "don't use trash" it gets ignored.

      If a special value (-1) is returned (or we use a new field?), FS instances can return to the shell saying "no need for trash". Then you can turn it on for a bucket by bucket basis; any store with the feature enabled can do the same thing.

      Alternative option: A special S3A-aware trash policy which does a getFileSystem.getConf.getBoolean("fs.s3a.trash.skip") & skips trash if set. This will all you to disable it on a bucket-by-bucket basis.

      Attachments

        Activity

          People

            Unassigned Unassigned
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: