[HDFS-1068] Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: namenode
• Assignee: Zhe Zhang
• Reporter: Hairong Kuang
• Created: 25/Mar/10 19:04
• Updated: 06/Sep/17 11:45

Description

In our production clusters, getFileInfo is the most frequent operation that hit NameNode, and its frequency is highly correlated to the GC behavior. HDFS-946 has already reduced the amount of heap/cpu and the number of temporary objects for each getFileInfo call. Yet another improvement is to avoid creation of a HdfsFileStatus object for each getFileInfo call. Instead each RPC handler can have a thread local HdfsFileStatus object. Each getFileInfo call simply sets values for all fields of the thread local HdfsFileStatus object.

Issue Links

• is related to: HDFS-1051 Umbrella Jira for Scaling the HDFS Name Service
• is related to: HDFS-12379 NameNode getListing should use FileStatus instead of HdfsFileStatus