[THRIFT-4626] Communication crash when using binary/compact protocol and zlib transport

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.0
• Fix Version/s: 0.12.0
• Component/s: Go - Library

Environment

Ubuntu 18.04

Go 1.10.1

• Assignee: James E. King III
• Reporter: Vyacheslav Kulakov
• Created: 31/Aug/18 21:54
• Updated: 27/Dec/18 15:25
• Resolved: 02/Sep/18 11:01

Description

Reading a request on the server side or reading a response on the client side always fail with the "Invalid data length" error when using the binary/compact protocol and the zlib transport, which wraps the framed transport.

In my project, I use the following code on the server side (only for testing):

processor := flume.NewThriftSourceProtocolProcessor(protocol)

serverTransport, _ := thrift.NewTServerSocketTimeout(address, timeout)

protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()
transportFactory := thrift.NewTFramedTransportFactory(thrift.NewTTransportFactory())
transportFactory = thrift.NewTZlibTransportFactoryWithFactory(level, transportFactory)

server := thrift.NewTSimpleServer4(
	processor,
	serverTransport,
	transportFactory,
	protocolFactory,
)

and following code on the client side:

factory := thrift.NewTBinaryProtocolFactoryDefault()
transport := thrift.TTransport(thrift.NewTFramedTransport(socket))
transport, err = thrift.NewTZlibTransport(transport, compressionLevel)
if err != nil {
	return err
}

err = transport.Open()
if err != nil {
	return err
}

client := flume.NewThriftSourceProtocolClient(
	thrift.NewTStandardClient(
		factory.GetProtocol(transport),
		factory.GetProtocol(transport),
	),
)

When I send data from the client to the server I always get the "EOF" error on the client and the "Invalid data length" error on the server. If I use the compact protocol the errors stay at their places.

I investigated Go library code and found a reason of that errors: the protocol invoke the RemainingBytes method on the zlib transport and it always return zero because all bytes were already read from a frame and were stored in a decompressor buffer. But we can't access to this buffer to obtain correct number of remaining bytes. So this combination of protocols and transports can't work together at all.

Issue Links

• links to: GitHub Pull Request #1589