[MAPREDUCE-2408] Make Gridmix emulate usage of data compression

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.23.0
• Component/s: contrib/gridmix
• Hadoop Flags: Reviewed

Release Note

Emulates the MapReduce compression feature in Gridmix. By default, compression emulation is turned on. Compression emulation can be disabled by setting 'gridmix.compression-emulation.enable' to 'false'. Use 'gridmix.compression-emulation.map-input.decompression-ratio', 'gridmix.compression-emulation.map-output.compression-ratio' and 'gridmix.compression-emulation.reduce-output.compression-ratio' to configure the compression ratios at map input, map output and reduce output side respectively. Currently, compression ratios in the range [0.07, 0.68] are supported. Gridmix auto detects whether map-input, map output and reduce output should emulate compression based on original job's compression related configuration parameters.

• Assignee: Amar Kamat
• Reporter: Ravi Gummadi
• Created: 29/Mar/11 09:29
• Updated: 15/Nov/11 00:48
• Resolved: 27/May/11 06:26

Description

Currently Gridmix emulates disk IO load only. This JIRA is to make Gridmix emulate load due to data compression as defined by the job-trace.

Issue Links

• is blocked by: MAPREDUCE-2153 Bring in more job configuration properties in to the trace file
• supercedes: MAPREDUCE-2542 [Gridmix] Add support for LZO codec in Gridmix