[HADOOP-16250] Add a way to declare a toplevel path in a non-auth FS as auth

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: fs/s3
• Assignee: Sean Mackrory
• Reporter: Steve Loughran
• Created: 11/Apr/19 20:10
• Updated: 19/Jul/19 18:28
• Resolved: 19/Jul/19 18:28

Description

Feature request from the Hive team. Add a way to declare that some part of a bucket is auth, while the rest of it isn't

This is pretty dangerous if you can start declaring arbitrary paths auth/non-auth, so here is my proposal

• bucket is considered non-auth
• only a single path is declared auth, after which all files under it will also be auth
• that path must be a single path element off root
• only operations which act within that path are auth.
• operations outside that path are non-auth
• anything which mixes (rename) is non-auth
• Lists on / are non-auth too.
• If a path is configured and it is not there, then everything is non-auth (i.e. its not an error, we just ignore the setting)

There's ~10 places in S3AFileSystem which could be changed from "isAuthoritative" to isAuthoritative(path); for rename we'd chedk src and dest

thoughts?

Issue Links

• is superceded by: HADOOP-16409 Allow authoritative mode on non-qualified paths