[HADOOP-2423] The codes in FSDirectory.mkdirs(...) is inefficient.

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.15.1
• Fix Version/s: 0.17.0
• Release Note: Improved FSDirectory.mkdirs(...) performance. In NNThroughputBenchmark-create, the ops per sec in was improved ~54%.
• Assignee: Tsz-wo Sze
• Reporter: Tsz-wo Sze
• Created: 13/Dec/07 23:45
• Updated: 12/Apr/11 23:38
• Resolved: 13/Mar/08 19:51

Description

FSDirectory.mkdirs(...) creates List<String> v to store all dirs. e.g.

//Suppose 
src = "/foo/bar/bas/"
//Then,
v = {"/", "/foo", "/foo/bar", "/foo/bar/bas"}

For each directory string cur in v, no matter cur already exists or not, it will try to do a unprotectedMkdir(cur, ...). Then, cur is parsed to byte[][] in INodeDirectory.addNode (...).

We don't need to do the parsing for each string in v. Instead, byte[][] should be stored. Also, the loop should not continue once it finds an existing subdirectory.

Issue Links

• is related to: HDFS-1832 FSNamesystem#startFileInternal unnecessarily traverses the directory multiple times
• relates to: HDFS-78 Eliminate redundant searches in the namespace directory tree.