[MAPREDUCE-4229] Counter names' memory usage can be decreased by interning

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.2, 2.0.2-alpha, 3.0.0-alpha1
• Fix Version/s: 2.0.3-alpha, 0.23.5
• Component/s: jobtracker
• Target Version/s: 0.23.5
• Hadoop Flags: Reviewed
• Assignee: Miomir Boljanovic
• Reporter: Todd Lipcon
• Created: 04/May/12 23:46
• Updated: 12/May/16 18:24
• Resolved: 23/Oct/12 21:22

Description

In our experience, most of the memory in production JTs goes to storing counter names (String objects and character arrays). Since most counter names are reused again and again, it would be a big memory savings to keep a hash set of already-used counter names within a job, and refer to the same object from all tasks.

Issue Links

• duplicates: MAPREDUCE-4303 Look at using String.intern to dedupe some Strings