[HDFS-13183] Standby NameNode process getBlocks request to reduce Active load

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.7.5, 3.1.0, 2.9.1, 2.8.4, 3.0.2
• Component/s: balancer & mover, namenode
• Assignee: Xiaoqiao He
• Reporter: Xiaoqiao He
• Created: 23/Feb/18 08:32
• Updated: 30/Dec/19 10:07

Description

The performance of Active NameNode could be impact when Balancer requests #getBlocks, since query blocks of overly full DNs performance is extremely inefficient currently. The main reason is NameNodeRpcServer#getBlocks hold read lock for long time. In extreme case, all handlers of Active NameNode RPC server are occupied by one reader NameNodeRpcServer#getBlocks and other write operation calls, thus Active NameNode enter a state of false death for number of seconds even for minutes.
The similar performance concerns of Balancer have reported by HDFS-9412, HDFS-7967, etc.

If Standby NameNode can shoulder #getBlocks heavy burden, it could speed up the progress of balancing and reduce performance impact to Active NameNode.

Issue Links

• is blocked by: HDFS-12976 Introduce ObserverReadProxyProvider
• relates to: HDFS-14162 Balancer should work with ObserverNode