Uploaded image for project: 'Thrift'
  1. Thrift
  2. THRIFT-1915 Multiplexing Services
  3. THRIFT-1914

Python: Support for Multiplexing Services on any Transport, Protocol and Server



    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.2
    • Python - Library
    • None
    • Patch Available


      Motivation and Benefits

      We plan to use Thrift with a large number of functions communicating among (at least) three languages. To keep maintainability high, we hope to avoid a single service defined with a large number of functions. This would require monolithic, unwieldy service implementations as the number of functions grows.

      Breaking up our API into multiple IDLs gives us multiple abstract base classes, which provides more flexibility in the object-oriented design of our server platform.

      Before our changes, the alternative was to open up additional ports with smaller service implementations on each. We'd rather not open additional ports.

      Our Approach

      We pursued an approach with the following in mind:

      • No modifications to existing Thrift code.
      • No modification to the Thrift protocol as described in the whitepaper.
      • No modification to any TServer, TProtocol or TTransport.
      • No need for any new TServer implementation. Works with any TServer implementation.
      • Work with any combination of TServer, TProtocol or TTransport.
      • Avoid language-specific features, to ease implementation in other languages.

      Additions to Thrift

      Convenience class:

      • TProtocolDecorator (extends TProtocol). This is a no-op decorator around the TProtocol abstract class.

      For use by clients:

      • TMultiplexedProtocol (extends TProtocolDecorator). This decorates any TProtocol by modifying the behaviour of writeMessageBegin(TMessage) to change TMessage.name from function_name to service_name + separator + function_name.

      For use by the server:

      • TMultiplexedProcessor (implements TProcessor). It should be used to communicate with a client that was written using TMultiplexedProtocol. It removes service_name + separator from TMessage.name, turning it back into the standard message format. It then brokers the service request to the TProcessor which is registered to handle requests for that service.

      Sample Usage - Client

      In this example, we've chosen to use TBinaryProtocol with two services: Calculator and WeatherReport.

      TSocket transport = new TSocket("localhost", 9090);
      TBinaryProtocol protocol = new TBinaryProtocol(transport);
      TMultiplexedProtocol mp = new TMultiplexedProtocol(protocol, "Calculator");
      Calculator.Client service = new Calculator.Client(mp);
      TMultiplexedProtocol mp2 = new TMultiplexedProtocol(protocol, "WeatherReport");
      WeatherReport.Client service2 = new WeatherReport.Client(mp2);

      Sample Usage - Server

      TMultiplexedProcessor processor = new TMultiplexedProcessor();
          new Calculator.Processor(new CalculatorHandler()));
          new WeatherReport.Processor(new WeatherReportHandler()));
      TServerTransport t = new TServerSocket(9090);
      TSimpleServer server = new TSimpleServer(processor, t);


        1. python_multiplex.patch
          7 kB
          Doug MacEachern
        2. python3_multiplex.patch
          3 kB

        Issue Links



              roger Roger Meier
              dougm Doug MacEachern
              2 Vote for this issue
              8 Start watching this issue