[HADOOP-18013] ABFS: add cloud trash policy with per-schema policy selection

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.2
• Component/s: fs/azure
• Labels: pull-request-available
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 18/Nov/21 14:37
• Updated: 02/Jul/24 10:45

Description

Add a custom TrashPolicy for azure which

• is patched in to the config automatically if "fs.trash.classname" is unset
• checks the source for existing at all before trying anything
• considers itself enabled even if the trash interval is 0
• if interval is 0, just does a delete (issue: what about 90s timeouts)
• downgrades exceptions/failures during rename. If anything is raised and the source dir isn't there, all is good.

Issue Links

• is related to: HADOOP-17691 Abfs directory delete times out on large directory tree w/ Oauth: OperationTimedOut
• is superceded by: HADOOP-19217 Introduce getTrashPolicy to FileSystem API
• links to: GitHub Pull Request #4729

Sub-Tasks

• Make Trash Policy pluggable for different FileSystems: Open