Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15703

Deadline scheduler needs to return to the client info about skipped calls, not just drop them

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.3.0
    • 1.3.0, 2.0.0
    • IPC/RPC, Scheduler
    • None
    • Reviewed
    • Hide
      With previous deadline mode of RPC scheduling (the implementation in SimpleRpcScheduler, which is basically a FIFO except that long-running scans are de-prioritized) and FIFO-based RPC scheduler clients are getting CallQueueTooBigException when RPC call queue is full.

      With this patch and when hbase.ipc.server.callqueue.type property is set to "codel" mode, clients will also be getting CallDroppedException, which means that the request was discarded by the server as it considers itself to be overloaded and starts to drop requests to avoid going down under the load. The clients will retry upon receiving this exception. It doesn't clear MetaCache with region locations.
      Show
      With previous deadline mode of RPC scheduling (the implementation in SimpleRpcScheduler, which is basically a FIFO except that long-running scans are de-prioritized) and FIFO-based RPC scheduler clients are getting CallQueueTooBigException when RPC call queue is full. With this patch and when hbase.ipc.server.callqueue.type property is set to "codel" mode, clients will also be getting CallDroppedException, which means that the request was discarded by the server as it considers itself to be overloaded and starts to drop requests to avoid going down under the load. The clients will retry upon receiving this exception. It doesn't clear MetaCache with region locations.

    Description

      In AdaptiveLifoCodelCallQueue we drop the calls when we think we're overloaded, we should instead return CallDroppedException to the cleent or something.

      Attachments

        1. HBASE-15703-branch-1.3.v1.patch
          8 kB
          Mikhail Antonov
        2. HBASE-15703-branch-1.3.v2.patch
          11 kB
          Mikhail Antonov

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. HBASE-15136 Explore different queuing behaviors while busy

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              mantonov Mikhail Antonov
              mantonov Mikhail Antonov
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: