[HADOOP-18878] remove/deprecate fs.s3a.multipart.purge

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.6
• Component/s: fs/s3
• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 01/Sep/23 16:36
• Updated: 01/Sep/23 16:36

Description

the fs.s3a.multipart.purge option has been in for a long time, to help avoid running up costs from incomplete uploads, especially during testing.

but

• adds overhead on startup (list + delete)
• dangerous if one client has a very short lifespan; all active s3a committer jobs will be broken
• obsoleted by s3 lifecycle rules
• and "hadoop s3guard uploads" cli.

proposed: deprecate for the next release, delete after.