[THRIFT-3768] TThreadedServer may crash if it is destroyed immediately after it returns from serve(); TThreadedServer disconnects clients when they connec

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.9.3
• Fix Version/s: 0.10.0
• Component/s: C++ - Library
• Assignee: James E. King III
• Reporter: Ted Wang
• Created: 31/Mar/16 16:35
• Updated: 12/Aug/16 01:29
• Resolved: 08/Jul/16 20:12

Description

Here's a sequence that shows the race:

Thread-1 (Users of TThreadedServer): Calls TThreadedServer::stop(), which calls interruptChildren and initiates the tearing down of client connections.

Thread-2: In TServerFramework::serve(), broke out of accept, and now blocks in TThreadedServer::serve() waiting to drain all the clients.

Thread-3 (The connected client thread created by TThreadedServer): In disposeConnectedClient, running because the server is shutting down and the shared_ptr specified this function to be the cleanup function for the client. This thread just returned from onClientDisconnected and now context switches.

Thread-2: TThreadedServer::serve() is notified that all of the clients have disconnected and completes.

Thread-1: Joins on Thread-2 and destroys the server object because it is done.

Thread-3: Finally gets a chance to run, but now encounters undefined behavior because it is still executing a member function of an object that has been destroyed.

You can force this race in action if you put sleep(1) before onClientDisconnected() in disposeConnectedClient

Issue Links

• relates to: THRIFT-3096 Consolidate TSimpleServer, TThreadedServer, TThreadPoolServer into one
• links to: GitHub Pull Request #977
• links to: GitHub Pull Request #980