[ARROW-16597] [Python][FlightRPC] Active server may segfault if Python interpreter shuts down

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 8.0.0
• Fix Version/s: 9.0.0
• Component/s: FlightRPC, Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/31952
• Assignee: David Li
• Reporter: David Li
• Created: 17/May/22 15:23
• Updated: 11/Jan/23 11:45
• Resolved: 31/May/22 16:27

Description

On Linux, this reliably segfaults for me with FATAL: exception not rethrown. Adding a server.shutdown to the end fixes it.

The reason is that the Python interpreter exits after running the script, and other Python threads call PyThread_exit_thread. But one of the Python threads is currently in the middle of executing the RPC handler. PyThread_exit_thread boils down to pthread_exit which works by throwing an exception that it expects will not be caught. But gRPC places a catch(...) around RPC handlers and catches this exception, and then pthreads aborts when it doesn't catch the exception.

We should force servers to shutdown at exit to avoid this.

import traceback
import pyarrow as pa
import pyarrow.flight as flight

class Server(flight.FlightServerBase):
    def do_put(self, context, descriptor, reader, writer):
        raise flight.FlightCancelledError("foo", extra_info=b"bar")


print("PyArrow version:", pa.__version__)
server = Server("grpc://localhost:0")
client = flight.connect(f"grpc://localhost:{server.port}")

schema = pa.schema([])
writer, reader = client.do_put(flight.FlightDescriptor.for_command(b""), schema)
try:
    writer.done_writing()
except flight.FlightError as e:
    traceback.print_exc()
    print(e.extra_info)
except Exception:
    traceback.print_exc()

Issue Links

• links to: GitHub Pull Request #13176