[HDFS-14408] HttpFS handles paths with special charactes different than WebHdfs

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.1.1
• Component/s: httpfs
• Assignee: Unassigned
• Reporter: Andrey Zinovyev
• Created: 03/Apr/19 19:06
• Updated: 09/Aug/19 21:54
• Resolved: 09/Aug/19 21:54

Description

After HDFS-13176 WebHdfsFileSystem encodes special characters twice. For example path /tmp/day=2018-01-01 becomes /webhdfs/v1/tmp/day%253D2018-01-01 call.

In NamenodeWebHdfsMethods it is handled by decode path twice (first by web server and then in code explicityly).

But if we use httpfs it fails to get paths with special characters (like `=`), cause it decodes path once.

Test to reproduce and simple fix in attachment. Although I think that double encoding doesn't look right.

Issue Links

• Is contained by: HDFS-14379 WebHdfsFileSystem.toUrl double encodes characters