Details
-
Documentation
-
Status: Resolved
-
Major
-
Resolution: Cannot Reproduce
-
1.4.0
-
None
-
None
Description
I was trying to setup a basic 2-tier Flume setup using the Avro source/sink to communicate between tiers. When I setup the system I used compression-type on the 1st tier's sink side like documented here (https://flume.apache.org/FlumeUserGuide.html#avro-sink)
agent.sinks.snk1.compression-type = deflate
And the same on the source side:
agent.sources.src1.compression-type = deflate
But I was getting this error:
08 Apr 2014 00:15:07,606 WARN pool-7-thread-1 (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught:201) - Unexpected exception from downstream.
org.jboss.netty.handler.codec.compression.CompressionException: decompression failure (-3): unknown compression method
at org.jboss.netty.handler.codec.compression.ZlibUtil.exception(ZlibUtil.java:31)
at org.jboss.netty.handler.codec.compression.ZlibUtil.fail(ZlibUtil.java:27)
at org.jboss.netty.handler.codec.compression.ZlibDecoder.decode(ZlibDecoder.java:153)
at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:67)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
After some digging I found if I changed 'deflate' for 'gzip' it works.
agent.sinks.snk1.compression-type = gzip
agent.sources.src1.compression-type = gzip