[HDFS-8865] Improve quota initialization performance

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0, 3.0.0-alpha1, 2.6.6, 2.7.5
• Target Version/s: 2.8.0
• Hadoop Flags: Reviewed
• Assignee: Kihwal Lee
• Reporter: Kihwal Lee
• Created: 06/Aug/15 18:00
• Updated: 02/Oct/19 17:12
• Resolved: 02/Oct/17 22:55

Description

After replaying edits, the whole file system tree is recursively scanned in order to initialize the quota. For big name space, this can take a very long time. Since this is done during namenode failover, it also affects failover latency.

By using the Fork-Join framework, I was able to greatly reduce the initialization time. The following is the test result using the fsimage from one of the big name nodes we have.

threads	seconds
1 (existing)	55
1 (fork-join)	68
4	16
8	8
12	6
16	5
20	4

Issue Links

• is duplicated by: HDFS-11254 Standby NameNode may crash during failover if loading edits takes too long
• is related to: HDFS-11254 Standby NameNode may crash during failover if loading edits takes too long
• relates to: HDFS-11192 OOM during Quota Initialization lead to Namenode hang
• relates to: HDFS-6763 Initialize file system-wide quota once on transitioning to active
• relates to: HDFS-9003 ForkJoin thread pool leaks
• relates to: HDFS-8879 Quota by storage type usage incorrectly initialized upon namenode restart