Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
Reviewed
Description
Command line options -Dmapred.output.compress and -Dmapred.output.compression.codec worked in Pig 0.7, which, when set, would compress the output, whether or not the output has an extension .gz, .bz, or .bz2. This behavior changed in 0.8 in that compression is on only if the output has such extensions. In other words, the command line options have no effect.
Pig needs to clarify the right way to enable/disable compression and implement it accordingly.
The behavior change is probably related to PIg-1533.