[HDFS-1583] Improve backup-node sync performance by wrapping RPC parameters

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.23.0
• Component/s: namenode
• Assignee: Liyin Liang
• Reporter: Liyin Liang
• Created: 13/Jan/11 05:47
• Updated: 22/Mar/11 06:01
• Resolved: 22/Mar/11 06:01

Description

The journal edit records are sent by the active name-node to the backup-node with RPC:

  public void journal(NamenodeRegistration registration,
                      int jAction,
                      int length,
                      byte[] records) throws IOException;

During the name-node throughput benchmark, the size of byte array records is around 8000. Then the serialization and deserialization is time-consuming. I wrote a simple application to test RPC with byte array parameter. When the size got to 8000, each RPC call need about 6 ms. While name-node sync 8k byte to local disk only need 0.3~0.4ms.

Issue Links

• is blocked by: HADOOP-6949 Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting
• relates to: HADOOP-4539 Streaming Edits to a Backup Node.