[THRIFT-5175] Maybe a race condition in TThreadedServerPool java implementation

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 0.13.0
• Component/s: Java - Library
• Assignee: Unassigned
• Reporter: Xiangdong Huang
• Created: 16/Apr/20 02:39
• Updated: 16/Apr/20 11:00
• Resolved: 16/Apr/20 11:00

Description

Hi, 

I notice THRIFT-1795 https://issues.apache.org/jira/browse/THRIFT-1795 solves a problem of the race condition in TThreadedServerPool.

 

However , now when I use Thrift v0.13.0, I occur the same problem..

 

Some codes:

The poolServer is TThreadPoolServer.

The serverTransport is TServerSocket.

I have a thread to run poolServer.serve().

And, another thread will call the following close() method to close the thrift.

 

// code placeholder
private synchronized void close() {
  if (poolServer != null) {
    poolServer.stop();
    poolServer = null;
  }
  if (serverTransport != null) {
    serverTransport.close();
    serverTransport = null;
  }
}

 

 

 

Then, after calling the above close() method, the console will print:

 

// code placeholder

WARN  [2020-04-16 02:14:46,097] [RPC-ServerServiceImpl] org.apache.thrift.server.TThreadPoolServer:230 - Transport error occurred during acceptance of message. 
org.apache.thrift.transport.TTransportException: No underlying server socket. 
at org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:126) 
at org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:35) 
at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:60) 
at org.apache.thrift.server.TThreadPoolServer.execute(TThreadPoolServer.java:185) 
at org.apache.thrift.server.TThreadPoolServer.serve(TThreadPoolServer.java:175) 
at org.apache.iotdb.db.service.RPCService$RPCServiceThread.run(RPCService.java:219)

WARN  [2020-04-16 02:14:46,103] [RPC-ServerServiceImpl] org.apache.thrift.server.TThreadPoolServer:230 - Transport error occurred during acceptance of message. 
org.apache.thrift.transport.TTransportException: No underlying server socket. 
at org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:126) 
at org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:35) 
at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:60) 
at org.apache.thrift.server.TThreadPoolServer.execute(TThreadPoolServer.java:185) 
at org.apache.thrift.server.TThreadPoolServer.serve(TThreadPoolServer.java:175) 
at org.apache.iotdb.db.service.RPCService$RPCServiceThread.run(RPCService.java:219)

WARN  [2020-04-16 02:14:46,105] [RPC-ServerServiceImpl] org.apache.thrift.server.TThreadPoolServer:230 - Transport error occurred during acceptance of message. 
org.apache.thrift.transport.TTransportException: No underlying server socket. 
at org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:126) 
at org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:35) 
at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:60) 
at org.apache.thrift.server.TThreadPoolServer.execute(TThreadPoolServer.java:185) 
at org.apache.thrift.server.TThreadPoolServer.serve(TThreadPoolServer.java:175) 
at org.apache.iotdb.db.service.RPCService$RPCServiceThread.run(RPCService.java:219)

This exception repeats infinitely.

 

 

The issues is hard to be reproduced on my MacOS, but can be reproduced on Travis-CI, in some environments (Linux-openjdk11, linux-openjdk8, win-openjdk11):

I attached a log file here (but maybe it can not give you more info).

 

I read the source code and feel that it is very strange. Thrift has marked the stopped_ field as volatile, I do not know why the execute() method of TThreadPoolServer does not break out the loop.