[THRIFT-1914] Python: Support for Multiplexing Services on any Transport, Protocol and Server

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.2
• Component/s: Python - Library
• Patch Info: Patch Available
• Assignee: Roger Meier
• Reporter: Doug MacEachern
• Created: 02/Apr/13 22:41
• Updated: 06/Jul/14 04:00
• Resolved: 03/May/14 15:54

Description

Motivation and Benefits

We plan to use Thrift with a large number of functions communicating among (at least) three languages. To keep maintainability high, we hope to avoid a single service defined with a large number of functions. This would require monolithic, unwieldy service implementations as the number of functions grows.

Breaking up our API into multiple IDLs gives us multiple abstract base classes, which provides more flexibility in the object-oriented design of our server platform.

Before our changes, the alternative was to open up additional ports with smaller service implementations on each. We'd rather not open additional ports.

Our Approach

We pursued an approach with the following in mind:

• No modifications to existing Thrift code.
• No modification to the Thrift protocol as described in the whitepaper.
• No modification to any TServer, TProtocol or TTransport.
• No need for any new TServer implementation. Works with any TServer implementation.
• Work with any combination of TServer, TProtocol or TTransport.
• Avoid language-specific features, to ease implementation in other languages.

Additions to Thrift

Convenience class:

• TProtocolDecorator (extends TProtocol). This is a no-op decorator around the TProtocol abstract class.

For use by clients:

• TMultiplexedProtocol (extends TProtocolDecorator). This decorates any TProtocol by modifying the behaviour of writeMessageBegin(TMessage) to change TMessage.name from function_name to service_name + separator + function_name.

For use by the server:

• TMultiplexedProcessor (implements TProcessor). It should be used to communicate with a client that was written using TMultiplexedProtocol. It removes service_name + separator from TMessage.name, turning it back into the standard message format. It then brokers the service request to the TProcessor which is registered to handle requests for that service.

Sample Usage - Client

In this example, we've chosen to use TBinaryProtocol with two services: Calculator and WeatherReport.

TSocket transport = new TSocket("localhost", 9090);
transport.open();

TBinaryProtocol protocol = new TBinaryProtocol(transport);

TMultiplexedProtocol mp = new TMultiplexedProtocol(protocol, "Calculator");
Calculator.Client service = new Calculator.Client(mp);

TMultiplexedProtocol mp2 = new TMultiplexedProtocol(protocol, "WeatherReport");
WeatherReport.Client service2 = new WeatherReport.Client(mp2);

System.out.println(service.add(2,2));
System.out.println(service2.getTemperature());

Sample Usage - Server

TMultiplexedProcessor processor = new TMultiplexedProcessor();

processor.registerProcessor(
    "Calculator",
    new Calculator.Processor(new CalculatorHandler()));

processor.registerProcessor(
    "WeatherReport",
    new WeatherReport.Processor(new WeatherReportHandler()));

TServerTransport t = new TServerSocket(9090);
TSimpleServer server = new TSimpleServer(processor, t);

server.serve();

Issue Links

• is a clone of: THRIFT-563 Support for Multiplexing Services on any Transport, Protocol and Server