[HADOOP-14114] S3A can no longer handle unencoded + in URIs

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.8.0
• Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
• Component/s: fs/s3
• Assignee: Sean Mackrory
• Reporter: Sean Mackrory
• Created: 23/Feb/17 19:28
• Updated: 08/Jun/17 00:13
• Resolved: 24/Feb/17 10:49

Description

Amazon secret access keys can include alphanumeric characters, but also / and + (I wish there was an official source that was really specific on what they can contain, but I'll have to rely on a few blog posts and my own experience).

Keys containing slashes used to be impossible to embed in the URL (e.g. s3a://access_key:secret_key@bucket/) but it is now possible to do it via URL encoding. Pluses used to work, but that is now only possible via URL encoding.

In the case of pluses, they don't appear to cause any other problems for parsing. So IMO the best all-around solution here is for people to URL-encode these keys always, but so that keys that used to work just fine can continue to work fine, all we need to do is detect that, log a warning, and we can re-encode it for the user.

Issue Links

• is broken by: HADOOP-3733 "s3:" URLs break when Secret Key contains a slash, even if encoded
• is depended upon by: HADOOP-13204 Über-jira: S3a phase III: scale and tuning