[HDFS-13693] Remove unnecessary search in INodeDirectory.addChild during image loading

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0, 3.1.4, 3.2.2
• Component/s: namenode
• Hadoop Flags: Reviewed
• Assignee: Lisheng Sun
• Reporter: zhouyingchao
• Created: 21/Jun/18 10:12
• Updated: 15/Oct/19 12:02
• Resolved: 23/Jul/19 03:27

Description

In FSImageFormatPBINode.loadINodeDirectorySection, all child INodes are added to their parent INode's map one by one. The adding procedure will search a position in the parent's map and then insert the child to the position. However, during image loading, the search is unnecessary since the insert position should always be at the end of the map given the sequence they are serialized on disk.

Test this patch against a fsimage of a 70PB  cluster (200million files and 300million blocks), the image loading time be reduced from 1210 seconds to 1138 seconds.So it can reduce up to about 10% of time.

Issue Links

• links to: GitHub Pull Request #1040