[HADOOP-18371] s3a FS init logs at warn if fs.s3a.create.storage.class is unset

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.3.5
• Fix Version/s: 3.3.5
• Component/s: fs/s3
• Labels: pull-request-available
• Assignee: Viraj Jasani
• Reporter: Steve Loughran
• Created: 27/Jul/22 11:00
• Updated: 16/Aug/22 18:50
• Resolved: 16/Aug/22 18:50

Description

if you don't have an s3a storage class set in fs.s3a.create.storage.class, then whenever you create an S3A FS instance, it logs at warn

bin/hadoop s3guard bucket-info $BUCKET

2022-07-27 11:53:11,239 [main] INFO  Configuration.deprecation (Configuration.java:logDeprecation(1459)) - fs.s3a.server-side-encryption.key is deprecated. Instead, use fs.s3a.encryption.key
2022-07-27 11:53:11,240 [main] INFO  Configuration.deprecation (Configuration.java:logDeprecation(1459)) - fs.s3a.server-side-encryption-algorithm is deprecated. Instead, use fs.s3a.encryption.algorithm
2022-07-27 11:53:11,396 [main] WARN  s3a.S3AFileSystem (S3AFileSystem.java:createRequestFactory(1004)) - Unknown storage class property fs.s3a.create.storage.class: ; falling back to default storage class
2022-07-27 11:53:11,839 [main] INFO  impl.DirectoryPolicyImpl (DirectoryPolicyImpl.java:getDirectoryPolicy(189)) - Directory markers will be kept
Filesystem s3a://stevel-london
Location: eu-west-2

note, this is why part of quaifying an sdk update involves looking at the logs and running the CLI commands by hand...you see if new messages have crept in

Issue Links

• is caused by: HADOOP-12020 Support configuration of different S3 storage classes
• links to: GitHub Pull Request #4730