Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18500

Performance issue: Don't use BufferedMutator for HTable's put method



    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      Remove the deprecated method get/setWriteBufferSize from Table and remove writeBufferSize from TableBuilder. Remove the BufferedMutatorImpl from HTable.


      Copied the test result from HBASE-17994.
      Run start-hbase.sh in my local computer and use the default config to test with PE tool.

      ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000 --nomapred --autoFlush=True randomWrite 1
      ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000 --nomapred --autoFlush=True asyncRandomWrite 1

      Mean latency test result.

        Test1 Test2 Test3 Test4 Test5
      randomWrite 164.39 161.22 164.78 140.61 151.69
      asyncRandomWrite 122.29 125.58 122.23 113.18 123.02

      50th latency test result.

        Test1 Test2 Test3 Test4 Test5
      randomWrite 130.00 125.00 123.00 112.00 121.00
      asyncRandomWrite 95.00 97.00 95.00 88.00 95.00

      99th latency test result.

        Test1 Test2 Test3 Test4 Test5
      randomWrite 600.00 600.00 650.00 404.00 425.00
      asyncRandomWrite 339.00 327.00 297.00 311.00 318.00

      In our internal 0.98 branch, the PE test result shows the async write has the almost same latency with the blocking write. But for master branch, the result shows the async write has better latency than the blocking client. Take a look about the code, I thought the difference is the BufferedMutator. For master branch, HTable don't have a write buffer and all write request will be flushed directly. And user can use BufferedMutator when user want to perform client-side buffering of writes. For the performance issue (autoFlush=True), I thought we can use rpc caller directly in HTable's put method. Thanks.

      Review: https://reviews.apache.org/r/61454/

      Copy the comments from Chia-Ping Tsai. Remove the BufferdMutator brings four benefits.
      1. correct the metrics (see HBASE-18476)
      2. make HTable thread-safe (see HBASE-17368)
      3. reduce the latency
      4. get rid of some deprecated methods in Table


        1. HBASE-18500-v6.patch
          41 kB
          Guanghao Zhang
        2. HBASE-18500-v5.patch
          39 kB
          Guanghao Zhang
        3. HBASE-18500-v5.patch
          39 kB
          Guanghao Zhang
        4. HBASE-18500-v5.patch
          39 kB
          Guanghao Zhang
        5. HBASE-18500-v4.patch
          38 kB
          Guanghao Zhang
        6. HBASE-18500-v3.patch
          35 kB
          Guanghao Zhang
        7. HBASE-18500-v2.patch
          31 kB
          Guanghao Zhang
        8. HBASE-18500-v1.patch
          4 kB
          Guanghao Zhang

          Issue Links



              • Assignee:
                zghao Guanghao Zhang
                zghao Guanghao Zhang
              • Votes:
                0 Vote for this issue
                15 Start watching this issue


                • Created: