-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 1.10.0
-
Component/s: Connectors / FileSystem
-
Labels:
When using CompressWriterFactory with Hadoop compression codec, the execution fails with java.io.NotSerializableException.
I guess this is probably to do with the the instance creation for Hadoop's CompressionCodec being done here at CompressWriterFactory.java#L59 and thus it has to be sent over the wire causing the exception to be thrown.
So I did a quick test on my end by changing the way the CompressionCodec is initialised and ran it on a Hadoop cluster, and it has been working just fine. Will raise a PR in a day or so.
- links to