[HADOOP-6460] Namenode runs of out of memory due to memory leak in ipc Server

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.20.1, 0.21.0, 0.22.0
• Fix Version/s: 0.20.2
• Release Note: If an IPC server response buffer has grown to than 1MB, it is replaced by a smaller buffer to free up the Java heap that was used. This will improve the longevity of the name service.
• Assignee: Suresh Srinivas
• Reporter: Suresh Srinivas
• Created: 19/Dec/09 17:49
• Updated: 02/Jul/10 04:46
• Resolved: 24/Dec/09 08:41

Description

Namenode heap usage grows disproportional to the number objects supports (files, directories and blocks). Based on heap dump analysis, this is due to large growth in ByteArrayOutputStream allocated in o.a.h.ipc.Server.Handler.run().

Issue Links

• is duplicated by: HADOOP-4802 RPC Server send buffer retains size of largest response ever sent
• relates to: HADOOP-6577 IPC server response buffer reset threshold should be configurable