Description
If the data that comes in through the Netty channel buffer is not framed correctly/is not valid Avro data, then the incoming data can cause arbitrarily large array lists to be created, causing OutOfMemoryError.
The relevant code(org.apache.avro.ipc.NettyTransportCodec):
private boolean decodePackHeader(ChannelHandlerContext ctx, Channel channel,
ChannelBuffer buffer) throws Exception {
if (buffer.readableBytes()<8)
int serial = buffer.readInt();
listSize = buffer.readInt();
dataPack = new NettyDataPack(serial, new ArrayList<ByteBuffer>(listSize));
return true;
}
If the buffer does not have valid Avro data, the listSize variable can have arbitrary values, causing massive ArrayLists to be created, leading to OutOfMemoryErrors.
Attachments
Attachments
Issue Links
- blocks
-
FLUME-1259 Flume throws OutOfMemory error when sending data from netcat to avro source (negative test case)
- Resolved
- is related to
-
AVRO-1226 Non-Avro data causes runtime exceptions/errors when sent to Avro NettyTransceiver
- Open