[MAPREDUCE-4354] Performance improvement with compressor object reinit restriction

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Invalid
• Affects Version/s: 0.20.205.0
• Fix Version/s: 0.20.205.0
• Component/s: performance
• Labels: performance
• Assignee: Unassigned
• Reporter: Ankit Kamboj
• Created: 19/Jun/12 22:43
• Updated: 21/Mar/13 20:11
• Resolved: 21/Mar/13 20:11

Description

HADOOP-5879 patch aimed at picking the conf (instead of default) settings for GzipCodec. It also involved re-initializing the recycled compressor object.
On our performance tests, this re-initialization led to performance degradation of 15% for LzoCodec because re-initialization for Lzo involves reallocation of buffers. LzoCodec takes the initial settings from config so it is not necessary to re-initialize it. This patch checks for the codec class and calls reinit only if the codec class is Gzip. This led to significant performance improvement of 15% for LzoCodec.