[PIG-4341] Add CMX support to pig.tmpfilecompression.codec

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.13.0
• Fix Version/s: 0.18.0
• Component/s: impl
• Assignee: fang fang chen
• Reporter: fang fang chen
• Created: 24/Nov/14 03:20
• Updated: 24/May/17 23:03

Description

Pig has supported compression(GZ, GZIP, LZO). But latest pig has not supported CMX codec yet. cmx is "com.ibm.biginsights.compress.CmxCodec". This information also could be found from latest release pig-0.13.0 documentation: http://pig.apache.org/docs/r0.13.0/perf.html.
Besides, I once tested CMX codec with pig-0.13.0 before. Following was the current settings:
SET pig.tmpfilecompression true;
SET pig.tmpfilecompression.codec cmx;
Error:
Caused by: java.io.IOException: Invalid temporary file compression codec [cmx]. Expected compression codecs for org.apache.pig.impl.io.TFileStorage are GZ,GZIP,LZO.

Issue Links

• is blocked by: HADOOP-11418 Property "io.compression.codec.lzo.class" does not work with other value besides default
• relates to: HADOOP-11644 Contribute CMX compression