[HDFS-14423] Percent (%) and plus (+) characters no longer work in WebHDFS - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.2.0, 3.1.2
Fix Version/s: 2.10.0, 3.3.0, 3.2.1, 3.1.3
Component/s: webhdfs
Labels:
None
Environment:

Ubuntu 16.04, but I believe this is irrelevant.

Description

The following commands with percent (%) no longer work starting with version 3.1:

$ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/%
$ hadoop/bin/hdfs dfs -cat webhdfs://localhost/%
cat: URLDecoder: Incomplete trailing escape (%) pattern

Also, plus (+ ) characters get turned into spaces when doing DN operations:

$ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/a+b
$ hadoop/bin/hdfs dfs -mkdir webhdfs://localhost/c+d
$ hadoop/bin/hdfs dfs -ls /
Found 4 items
-rw-r--r--   1 jing supergroup          0 2019-04-12 11:20 /a b
drwxr-xr-x   - jing supergroup          0 2019-04-12 11:21 /c+d

I can confirm that these commands work correctly on 2.9 and 3.0. Also, the usual hdfs:// client works as expected.

I suspect a relation with ~~HDFS-13176~~ or ~~HDFS-13582~~, but I'm not sure what the right fix is. Note that Hive uses % to escape special characters in partition values, so banning % might not be a good option. For example, Hive will create a paths like table_name/partition_key=%2F when partition_key='/'.