[PIG-1714] Option mapred.output.compress doesn't work in Pig 0.8 but worked in 0.7

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.8.0
• Hadoop Flags: Reviewed
• Assignee: Xuefu Zhang
• Reporter: Xuefu Zhang
• Created: 10/Nov/10 18:40
• Updated: 21/Apr/11 20:38
• Resolved: 13/Nov/10 00:07

Description

Command line options -Dmapred.output.compress and -Dmapred.output.compression.codec worked in Pig 0.7, which, when set, would compress the output, whether or not the output has an extension .gz, .bz, or .bz2. This behavior changed in 0.8 in that compression is on only if the output has such extensions. In other words, the command line options have no effect.

Pig needs to clarify the right way to enable/disable compression and implement it accordingly.

The behavior change is probably related to PIg-1533.

Issue Links

• is related to: PIG-1791 System property mapred.output.compress, but pig-cluster-hadoop-site.xml doesn't
• is related to: PIG-1814 mapred.output.compress in SET statement does not work