[HIVE-2988] Use of XMLEncoder to serialize MapredWork causes OOM in hive cli

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: CLI
• Labels: Performance
• Assignee: Unassigned
• Reporter: Rohini Palaniswamy
• Created: 27/Apr/12 15:37
• Updated: 12/Jan/13 21:24

Description

When running queries on tables with 6000 partitions, hive cli if configured with 128M runs into OOM. Heapdump showed 37MB occupied by one XMLEncoder object while the MapredWork was 500K which is highly inefficient. We should switch to using something more efficient like XStream.