[HDFS-16245] Record the number of INodeDirectory when loading the FSImage file

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Component/s: namenode
• Labels: pull-request-available
• Assignee: JiangHua Zhu
• Reporter: JiangHua Zhu
• Created: 30/Sep/21 12:15
• Updated: 23/Jul/22 08:32

Description

When starting parallel loading of FsImage (dfs.image.parallel.load=true), you can see in the startup log that the number of loaded INodes has been printed, but it is impossible to know the specific number of loaded INodeDirectory, because it is not printed now.
For example, here are some startup information:
2021-09-30 19:12:55,012 [15587]-INFO [main:FSImageFormatProtobuf$Loader@340]-The fsimage will be loaded in parallel using 4 threads
2021-09-30 19:12:55,031 [15606]-INFO [main:FSImageFormatPBINode$Loader@418]-Loading the INode section in parallel with 12 sub-sections
2021-09-30 19:12:55,034 [15609]-INFO [main:FSImageFormatPBINode$Loader@409]-Loading xxxx INodes.
......
2021-09-30 19:30:37,080 [1077655]-INFO [main:FSImageFormatPBINode$Loader@465]-Completed loading all INode sections. Loaded xxxx inodes.
2021-09-30 19:30:37,086 [1077661]-INFO [main:FSImageFormatPBINode$Loader@222]-Loading the INodeDirectory section in parallel with 12 sub-sections
......
2021-09-30 19:36:58,074 [1458649]-INFO [main:FSImageFormatPBINode$Loader@261]-Completed loading all INodeDirectory sub-sections
2021-09-30 19:36:58,076 [1458651]-INFO [main:FSImageFormatPBINode$Loader@339]-Completed update blocks map and name cache, total waiting duration 1 ms.
2021-09-30 19:36:58,111 [1458686]-INFO [main:FSImageFormatProtobuf$Loader@248]-Loaded FSImage in xxxx seconds.

Issue Links

• links to: GitHub Pull Request #3512