Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-13889

[C++][FlightRPC] Decouple Arrow Flight RPC from GRPC

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 5.0.0
    • None
    • C++, FlightRPC

    Description

      I'm trying to implement Flight RPC on RPC framework with protobuf message support in distributed system.

      However, the flight rpc is tied to grpc.
      Classes from grpc used in flight server are:
      1. `grpc::ServerContext` used in grpc generated code in parameter, and used to generate `ServerCallContext`.
      2. `grpc::Status` used in grpc generated code as return type.
      3. `grpc::ServerReaderWriter` and `grpc::ServerReader` used in massive wrapped MessageReader/Writer classes.

      1 & 2 are not coupled much with flight, while the third part is the tough work.
      Shall we introduce an interface class with same semantics to allow anyone implement the writing process to stream, such as `arrow::flight::ServerReaderWriter` and `arrow::flight::ServerReader`.

      So that, making a shim layer between `FlightServiceImpl` and `FlightServerBase` is possible to decouple flight from grpc, meanwhile taking advantage of its zero-copy messages.
      All message converting processes can be handled in the shim layer.
      For example, the function definition of `DoGet` can be `arrow::Status DoGet(ServerCallContext* context, const pb::Ticket* request, ServerWriter<pb::FlightData>* writer)`, which converts pb messages to flight's and call functions from actual business logic implementation from `FlightServerBase` as `Status DoGet(const ServerCallContext& context, const Ticket& request, std::unique_ptr<FlightDataStream>* stream)`.

      While, the client seems more complex, since the cookie stuff and others.
      If the idea above is possible, I'll have a exploration on client in depth

      Attachments

        Activity

          People

            Unassigned Unassigned
            hu6360567 Wenbo Hu
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m