HBase
  1. HBase
  2. HBASE-10605

Manage the call timeout in the server

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.99.0
    • Fix Version/s: None
    • Component/s: IPC/RPC, regionserver
    • Labels:
      None

      Description

      Since HBASE-10566, we have an explicit call timeout available in the client.
      We could forward it to the server, and use this information for:

      • if the call is still in the queue, just cancel it
      • if the call is under execution, makes this information available in RpcCallContext (actually change the RpcCallContext#disconnectSince to something more generic), so it can be used by the query under execution to stop its execution
      • in the future, interrupt it to manage the case 'stuck on a dead datanode' or something similar
      • if the operation has finished, don't send the reply to the client, as by definition the client is not interested anymore.

      From this, it will be easy to manage the cancellation: disconnect/timeout/cancellation are similar from a service execution PoV

        Issue Links

          Activity

          Hide
          Yong Zhang added a comment -

          Hi Nicolas Liochon, can please you give more info for handle client retrying and server timeout behavior?

          Show
          Yong Zhang added a comment - Hi Nicolas Liochon , can please you give more info for handle client retrying and server timeout behavior?
          Hide
          Nicolas Liochon added a comment -

          Hi Yong Zhang, what do you need to know?
          The point of this jira if that the server should not continue to handle a request if we know that the client has already stopped waiting for the result.

          Show
          Nicolas Liochon added a comment - Hi Yong Zhang , what do you need to know? The point of this jira if that the server should not continue to handle a request if we know that the client has already stopped waiting for the result.
          Hide
          Yong Zhang added a comment -

          How server side know client call timeout, I think client rpc should include the timeout parameter and submit time, and server side will have a monitor.
          And more hbase rpc client has retry, request time out seems after some calls failed or socket timeout, how to handle this scenario?

          Show
          Yong Zhang added a comment - How server side know client call timeout, I think client rpc should include the timeout parameter and submit time, and server side will have a monitor. And more hbase rpc client has retry, request time out seems after some calls failed or socket timeout, how to handle this scenario?
          Hide
          Nicolas Liochon added a comment -

          > , I think client rpc should include the timeout parameter
          Yes, we would need to forward the timeout (not the the submit time, because we don't want to rely on having the server and client clocks in sync: the server can use its own clock).
          Then there is already a check in the server, the request is cancelled if the client is disconnected (i.e. the tcp connection is closed).

          Show
          Nicolas Liochon added a comment - > , I think client rpc should include the timeout parameter Yes, we would need to forward the timeout (not the the submit time, because we don't want to rely on having the server and client clocks in sync: the server can use its own clock). Then there is already a check in the server, the request is cancelled if the client is disconnected (i.e. the tcp connection is closed).

            People

            • Assignee:
              Unassigned
              Reporter:
              Nicolas Liochon
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:

                Development