[HDFS-362] FSEditLog should not writes long and short as UTF8 and should not use ArrayWritable for writing non-array items

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0, 2.0.0-alpha
• Fix Version/s: 0.23.1
• Component/s: namenode
• Hadoop Flags: Reviewed
• Assignee: Uma Maheswara Rao G
• Reporter: Tsz-wo Sze
• Created: 05/May/08 23:35
• Updated: 10/Mar/15 04:35
• Resolved: 17/Sep/11 10:55

Description

In FSEditLog,

• long and short are first converted to String and are further converted to UTF8

• For some non-array items, it first create an ArrayWritable object to hold all the items and then writes the ArrayWritable object.

These result creating many intermediate objects which affects Namenode CPU performance and Namenode restart.

Issue Links

• is blocked by: HADOOP-7493 [HDFS-362] Provide ShortWritable class in hadoop.
• relates to: HADOOP-3248 Improve Namenode startup performance