[HADOOP-17928] s3a: set fs.s3a.downgrade.syncable.exceptions = true by default

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.1
• Fix Version/s: 3.3.2
• Component/s: fs/s3
• Labels: pull-request-available
• Target Version/s: 3.3.5
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 21/Sep/21 17:15
• Updated: 09/Nov/21 13:56
• Resolved: 09/Nov/21 13:56

Description

HADOOP-17597 set policy of reacting to hsync() on an s3 output stream to be one of :Fail, warn, with default == fail.

I propose downgrading this to warn. We've done it internally, after having it on fail long enough to identify which processes were doing either of

• having unrealistic expectations about the output stream (fix: move off s3)
• were using hflush() as a variant of flush(), with the failure being an over-reaction

Issue Links

• relates to: HADOOP-17597 Add option to downgrade S3A rejection of Syncable to warning
• links to: GitHub Pull Request #3585