[HADOOP-16343] Add option for a prefix to put in front of every s3guard table

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 3.2.0
• Component/s: fs/s3
• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 03/Jun/19 08:53
• Updated: 27/Feb/21 11:10
• Resolved: 27/Feb/21 11:10

Description

with DDB on demand, there's no cost in $ to having a table per bucket, and some security isolation benefits. But if you tell applications they can create DDB tables for every s3 bucket they encounter, their names may clash with other DDB tables, and it becomes hard to identify which tables in a region are s3.

Proposed: add a fs.s3a.s3guard.table.prefix option which lets you define the prefix to put in front of any inferred table name, so providing the isolation

note, AWS DDB naming rules put a limit of 255 chars; table prefix should short, e.g. "s3a."

Issue Links

• is related to: HADOOP-16252 Use configurable dynamo table name prefix in S3Guard tests