Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-16224

Reduce the number of RPCs for the large PUTs

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.0
    • 1.4.0, 2.0.0
    • Client
    • None
    • Reviewed

    Description

      This patch is proposed to reduce the number of RPC for the large PUTs

      The number and data size of write thread(SingleServerRequestRunnable) is a result of three main factors：
      1) The flush size taken by BufferedMutatorImpl#backgroundFlushCommits
      2) The limit of task number
      3) ClientBackoffPolicy

      A lot of requests created with less MUTATIONs is a result of two reason:
      1) many regions of target table are in different server.
      2) flush size in step one is summed by “all” server rather than “individual” server

      This patch removes the limit of flush size in step one and add maximum size to submit for each server in the AsyncProcess

      Attachments

        1. IntegrationTestBigLinkedList with chaos monkey (unbalance).txt
          70 kB
          Chia-Ping Tsai
        2. IntegrationTestBigLinkedList with chaos monkey (slowDeterministic).txt
          37 kB
          Chia-Ping Tsai
        3. IntegrationTestBigLinkedList with chaos monkey (serverKilling) (with backup master).txt
          66 kB
          Chia-Ping Tsai
        4. IntegrationTestBigLinkedList with chaos monkey (serverKilling).txt
          369 kB
          Chia-Ping Tsai
        5. HBASE-16224-v9.patch
          58 kB
          Chia-Ping Tsai
        6. HBASE-16224-v8.patch
          52 kB
          Chia-Ping Tsai
        7. HBASE-16224-v7.patch
          53 kB
          Chia-Ping Tsai
        8. HBASE-16224-v6.patch
          52 kB
          Chia-Ping Tsai
        9. HBASE-16224-v5.patch
          52 kB
          Chia-Ping Tsai
        10. HBASE-16224-v4.patch
          52 kB
          Chia-Ping Tsai
        11. HBASE-16224-v3.patch
          24 kB
          Chia-Ping Tsai
        12. HBASE-16224-v2.patch
          23 kB
          Chia-Ping Tsai
        13. HBASE-16224-v1.patch
          23 kB
          Chia-Ping Tsai
        14. HBASE-16224.v10.patch
          59 kB
          Chia-Ping Tsai
        15. HBASE-16224.branch-1.v9.patch
          50 kB
          Chia-Ping Tsai
        16. HBASE-16224.branch-1.v10.patch
          58 kB
          Chia-Ping Tsai
        17. experiment-v9.patch.xlsx
          87 kB
          Chia-Ping Tsai
        18. experiment.xlsx
          125 kB
          Chia-Ping Tsai

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. HBASE-20017 BufferedMutatorImpl submit the same mutation repeatedly

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          links to

          Web Link review board

          Activity

            People

              chia7712 Chia-Ping Tsai
              chia7712 Chia-Ping Tsai
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: