Description
A seemingly recurrent problem with s3guard is "people who think S3Guard is turned on but really it isn't"
It's not immediately obvious this is the case, and the fact S3Guard is off tends to surface after some intermittent failure has actually been detected.
Propose: add a configuration parameter which chooses what to do when an S3A FS is instantiated without S3Guard
- silent : today; do nothing.
- status: give s3guard on/off status
- inform: log FS is instantiated without s3guard
- warn: Warn that data may be at risk in workflows
- fail
deployments could then choose which level of reaction they want. I'd make the default "inform" for now; any on-prem object store deployment should switch to silent, and if you really want strictness, fail is the ultimate option
Attachments
Issue Links
- relates to
-
HADOOP-16953 HADOOP-16953. tune s3guard disabled warnings
- Resolved
- links to