[HADOOP-6949] Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.22.0
• Fix Version/s: 0.22.0, 0.23.0
• Component/s: io
• Hadoop Flags: Incompatible change, Reviewed

Release Note

Increments the RPC protocol version in org.apache.hadoop.ipc.Server from 4 to 5.
Introduces ArrayPrimitiveWritable for a much more efficient wire format to transmit arrays of primitives over RPC. ObjectWritable uses the new writable for array of primitives for RPC and continues to use existing format for on-disk data.

• Tags: Writable, primitive arrays, RPC
• Assignee: Matthew Foley
• Reporter: Navis Ryu
• Created: 10/Sep/10 08:49
• Updated: 02/May/13 02:29
• Resolved: 21/Mar/11 21:14

Description

Current implementation of oah.io.ObjectWritable marshals primitive array types as general object array ; array type string + array length + (element type string + value)*n

It would not be needed to specify each element types for primitive arrays.

Issue Links

• blocks: HDFS-1583 Improve backup-node sync performance by wrapping RPC parameters
• is depended upon by: HDFS-2126 Improve Namenode startup time [umbrella task]
• is related to: HADOOP-7158 Reduce RPC packet size for homogeneous arrays, such as the array responses to listStatus() and getBlockLocations()