[HDFS-14466] Add a regression test for HDFS-14323

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.3.0, 3.2.1, 3.1.3
• Component/s: fs, test, webhdfs
• Hadoop Flags: Reviewed
• Assignee: Masatake Iwasaki
• Reporter: Yuya Ebihara
• Created: 16/Apr/19 04:24
• Updated: 12/Jul/19 01:26
• Resolved: 12/Jul/19 01:26

Description

Recently, we upgraded hadoop library from 2.7.7 to 3.2.0. This issue happens after the update. When we call FileSystem.listLocatedStatus with location 'webhdfs://hadoop-master:50070/user/hive/warehouse/test_part/dt=1', the internal calls are

• 2.7.7 http://hadoop-master:50070/webhdfs/v1/user/hive/warehouse/test_part/dt=1?op=LISTSTATUS&user.name=xxx
• 3.2.0 http://hadoop-master:50070/webhdfs/v1/user/hive/warehouse/test_part/dt%253D1?op=LISTSTATUS&user.name=xxx'

As a result, it returns RemoteException with FileNotFoundException.

{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File /user/hive/warehouse/test_part/dt%3D1 does not exist."}}

Could you please tell me whether it's a bug and the way to avoid it?

Issue Links

• is broken by: HDFS-13176 WebHdfs file path gets truncated when having semicolon (;) inside
• is related to: HDFS-14323 Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path
• is related to: HDFS-13582 Improve backward compatibility for HDFS-13176 (WebHdfs file path gets truncated when having semicolon (;) inside)