[HADOOP-9953] Improve RPC server throughput

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.0.0-alpha, 3.0.0-alpha1
• Component/s: ipc
• Assignee: Daryn Sharp
• Reporter: Daryn Sharp
• Created: 11/Sep/13 21:55
• Updated: 12/May/16 18:24

Description

Bottlenecks in the RPC layer are in part holding back the performance of the NN. Even under very heavy load, the NN usually can't saturate more than a few cores even with load patterns dominated by read ops.

This will be an umbrella for issues discovered.

Issue Links

• relates to: HDFS-5238 Improve NN operation throughput

Sub-Tasks

• RPC idle connection closing is extremely inefficient: Closed
• RPC listener inefficiently assigns connections to readers: Closed
• Server.getNumOpenConnections may throw NPE: Closed
• backport hadoop-10107 to branch-0.23: Closed
• Allowed deferred sending of call responses: Resolved