Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15620 Über-jira: S3A phase VI: Hadoop 3.3 features
  3. HADOOP-15575

Add a way for an FS instance to say "really, no trash interval at all"

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.0
    • Fix Version/s: None
    • Component/s: fs, fs/s3
    • Labels:
      None

      Description

      Object stores like S3 often offer object versioning as a way to recover from accidental deletions. There is no need to use the Fs Shell trash mechanism.

      Proposed: a way for an FS instance to tell the shell to not use trash, even if the client has configured it. The current getServerDefaults().getTrashInterval() returns a value from the server, but it only overrides the ""fs.trash.interval" if it !=0. So if a server says "don't use trash" it gets ignored.

      If a special value (-1) is returned (or we use a new field?), FS instances can return to the shell saying "no need for trash". Then you can turn it on for a bucket by bucket basis; any store with the feature enabled can do the same thing.

      Alternative option: A special S3A-aware trash policy which does a getFileSystem.getConf.getBoolean("fs.s3a.trash.skip") & skips trash if set. This will all you to disable it on a bucket-by-bucket basis.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: