[HDFS-15052] WebHDFS getTrashRoot leads to OOM due to FileSystem object creation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.9.0, 3.0.0-alpha2
• Fix Version/s: 3.3.0, 3.1.4, 3.2.2, 2.10.1
• Component/s: webhdfs
• Hadoop Flags: Reviewed
• Assignee: Masatake Iwasaki
• Reporter: Wei-Chiu Chuang
• Created: 11/Dec/19 17:34
• Updated: 21/Feb/20 05:39
• Resolved: 21/Feb/20 05:39

Description

Quoting daryn in HDFS-10756 :

Surprised nobody has discovered this will lead to an inevitable OOM in the NN. The NN should not be creating filesystems to itself, and must never create filesystems in a remote user's context or the cache will explode.

I guess the problem lies in side NamenodeWebHdfsMethods#getTrashRoot

	  private static String getTrashRoot(String fullPath,
	      Configuration conf) throws IOException {
	    FileSystem fs = FileSystem.get(conf != null ? conf : new Configuration());
	    return fs.getTrashRoot(
	        new org.apache.hadoop.fs.Path(fullPath)).toUri().getPath();
	  }

Issue Links

• duplicates: HDFS-15181 Webhdfs getTrashRoot() causes internal AccessControlException
• is related to: HDFS-10756 Expose getTrashRoot to HTTPFS and WebHDFS
• relates to: HDFS-15181 Webhdfs getTrashRoot() causes internal AccessControlException
• links to: GitHub Pull Request #1758