Uploaded image for project: 'Thrift'
  1. Thrift
  2. THRIFT-4626

Communication crash when using binary/compact protocol and zlib transport

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.12.0
    • Component/s: Go - Library
    • Labels:
      None
    • Environment:

      Ubuntu 18.04

      Go 1.10.1

      Description

      Reading a request on the server side or reading a response on the client side always fail with the "Invalid data length" error when using the binary/compact protocol and the zlib transport, which wraps the framed transport.

      In my project, I use the following code on the server side (only for testing):

      processor := flume.NewThriftSourceProtocolProcessor(protocol)
      
      serverTransport, _ := thrift.NewTServerSocketTimeout(address, timeout)
      
      protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()
      transportFactory := thrift.NewTFramedTransportFactory(thrift.NewTTransportFactory())
      transportFactory = thrift.NewTZlibTransportFactoryWithFactory(level, transportFactory)
      
      server := thrift.NewTSimpleServer4(
      	processor,
      	serverTransport,
      	transportFactory,
      	protocolFactory,
      )
      

      and following code on the client side:

      factory := thrift.NewTBinaryProtocolFactoryDefault()
      transport := thrift.TTransport(thrift.NewTFramedTransport(socket))
      transport, err = thrift.NewTZlibTransport(transport, compressionLevel)
      if err != nil {
      	return err
      }
      
      err = transport.Open()
      if err != nil {
      	return err
      }
      
      client := flume.NewThriftSourceProtocolClient(
      	thrift.NewTStandardClient(
      		factory.GetProtocol(transport),
      		factory.GetProtocol(transport),
      	),
      )
      

      When I send data from the client to the server I always get the "EOF" error on the client and the "Invalid data length" error on the server. If I use the compact protocol the errors stay at their places.

      I investigated Go library code and found a reason of that errors: the protocol invoke the RemainingBytes method on the zlib transport and it always return zero because all bytes were already read from a frame and were stored in a decompressor buffer. But we can't access to this buffer to obtain correct number of remaining bytes. So this combination of protocols and transports can't work together at all.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jking3 James E. King III
                Reporter:
                vykulakov Vyacheslav Kulakov
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: