HBase
  1. HBase
  2. HBASE-5600

Make Endpoint Coprocessors Available from Thrift

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Thrift
    • Labels:

      Description

      Currently, the only way to access an endpoint coprocessor via thrift is to modify the schema and Thrift server for every coprocessor function. This is annoying. It should be possible to use your coprocessors without having to mangle HBase core code (since that's the point of coprocessors).

        Activity

        Ben West created issue -
        Hide
        Ben West added a comment -

        I'm not entirely sure how to accomplish this in full generality. The most obvious methods would involve sending java code through thrift, which is probably a no-go. However, here is one simple coprocessor which I have that it seems could be handled:

        Batch.Call call = Batch.forMethod(IMyEndpoint.class, "getRow", rowKey.getBytes());
        Map<byte[], byte[]> results = table.coprocessorExec(IMyEndpoint.class, null, null, call);
        

        We could create a thrift method to take the name of the class, method, and an array of params and then call coprocessorExec. If this sounds reasonable, I can try to allocate some time for a patch.

        Alternatively, we could just require that you write a client for your endpoints, and then thrift calls the client.

        Show
        Ben West added a comment - I'm not entirely sure how to accomplish this in full generality. The most obvious methods would involve sending java code through thrift, which is probably a no-go. However, here is one simple coprocessor which I have that it seems could be handled: Batch.Call call = Batch.forMethod(IMyEndpoint.class, "getRow" , rowKey.getBytes()); Map< byte [], byte []> results = table.coprocessorExec(IMyEndpoint.class, null , null , call); We could create a thrift method to take the name of the class, method, and an array of params and then call coprocessorExec. If this sounds reasonable, I can try to allocate some time for a patch. Alternatively, we could just require that you write a client for your endpoints, and then thrift calls the client.
        Hide
        Andrew Purtell added a comment -

        We could create a thrift method to take the name of the class, method, and an array of params and then call coprocessorExec.

        This sounds like a reasonable short term thing to do.

        For now with the dynamic behaviors of the current HRPC based stack we can mostly get away with using the same Java client tools for flexible remote method invocation of Endpoints as with the core interfaces. In the future the fact every Endpoint is really its own little protocol may be more exposed. In this world, the interface passes a blob. Such blobs could recursively contain protobuf (or Thrift) encoding.

        If going forward we will support Thrift and protobuf ("new RPC") clients both, then maybe we can expect server side code will translate from Thrift and protobuf message representations to some common representation, POJO or whatever. In other words, rehydrate from message representation into real classes (via reflection?) At least for Java, protobufs documentation recommends the objects built by the protobuf unmarshaller not be used directly as application classes. I think Thrift has the same practice. So on the server side that might not be so bad.

        On the client side, given the static nature of Thrift and protobuf message schemas (compiled from IDL) we can't dynamically create messages, so there's no way to hide behind for example a remote invocation proxy or some message builder. It could be different if we used Avro or some other option which can create message schemas at runtime and use those dynamically generated schemas server side.

        Show
        Andrew Purtell added a comment - We could create a thrift method to take the name of the class, method, and an array of params and then call coprocessorExec. This sounds like a reasonable short term thing to do. For now with the dynamic behaviors of the current HRPC based stack we can mostly get away with using the same Java client tools for flexible remote method invocation of Endpoints as with the core interfaces. In the future the fact every Endpoint is really its own little protocol may be more exposed. In this world, the interface passes a blob. Such blobs could recursively contain protobuf (or Thrift) encoding. If going forward we will support Thrift and protobuf ("new RPC") clients both, then maybe we can expect server side code will translate from Thrift and protobuf message representations to some common representation, POJO or whatever. In other words, rehydrate from message representation into real classes (via reflection?) At least for Java, protobufs documentation recommends the objects built by the protobuf unmarshaller not be used directly as application classes. I think Thrift has the same practice. So on the server side that might not be so bad. On the client side, given the static nature of Thrift and protobuf message schemas (compiled from IDL) we can't dynamically create messages, so there's no way to hide behind for example a remote invocation proxy or some message builder. It could be different if we used Avro or some other option which can create message schemas at runtime and use those dynamically generated schemas server side.
        Lars George made changes -
        Field Original Value New Value
        Labels thrift thrift thrift2

          People

          • Assignee:
            Unassigned
            Reporter:
            Ben West
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development