Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14331

a single callQueue related improvements

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Abandoned
    • None
    • None
    • IPC/RPC, Performance, Scheduler
    • None

    Description

      LinkedBlockingQueue well separates locks between the take method and the put method, but not between takers, and not between putters. These methods are implemented to take locks at the almost beginning of their logic. HBASE-11355 introduces multiple call-queues to reduce such possible congestion, but I doubt that it is required to stick to BlockingQueue.

      There are the other shortcomings of using BlockingQueue. When using multiple queues, since BlockingQueue blocks threads it is required to prepare enough threads for each queue. It is possible that there is a queue starving for threads while there is another queue where threads are idle. Even if you can tune parameters to avoid such situations, the tuning is not so trivial.

      I suggest using a single ConcurrentLinkedQueue with Semaphore.

      Attachments

        1. SemaphoreBasedPriorityBlockingQueue.java
          7 kB
          Hiroshi Ikeda
        2. SemaphoreBasedLinkedBlockingQueue.java
          1 kB
          Hiroshi Ikeda
        3. SemaphoreBasedBlockingQueue.java
          9 kB
          Hiroshi Ikeda
        4. HBASE-14331-V6.patch
          67 kB
          Hiroshi Ikeda
        5. HBASE-14331-V6.patch
          67 kB
          Hiroshi Ikeda
        6. HBASE-14331-V5.patch
          67 kB
          Hiroshi Ikeda
        7. HBASE-14331-V4.patch
          62 kB
          Hiroshi Ikeda
        8. HBASE-14331-V3.patch
          59 kB
          Hiroshi Ikeda
        9. HBASE-14331-V2.patch
          59 kB
          Hiroshi Ikeda
        10. HBASE-14331.patch
          55 kB
          Hiroshi Ikeda
        11. HBASE-14331.patch
          55 kB
          Michael Stack
        12. CallQueuePerformanceTestApp.java
          5 kB
          Hiroshi Ikeda
        13. BlockingQueuesPerformanceTestApp-output.txt
          3 kB
          Hiroshi Ikeda
        14. BlockingQueuesPerformanceTestApp-output.pdf
          254 kB
          Hiroshi Ikeda
        15. BlockingQueuesPerformanceTestApp.java
          5 kB
          Hiroshi Ikeda

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ikeda Hiroshi Ikeda
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: