Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24754

Bulk load performance is degraded in HBase 2

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.2.3, 2.5.0, 3.0.0-alpha-2, 2.6.0
    • None
    • Performance
    • None

    Description

      in our Test,It is observed that Bulk load performance is degraded in HBase 2 .

       Test Input: 

      1: Table with 500 region(300 column family)

      2:  data =2 TB

      Data Sample

      18600000001201502051000000068110,18600000001,20150205,5,404,735412,2938,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111,111111111111111111111111111111111111111111111111111111111111111111111111111111111

      3: Cluster: 7 node(2 master+5 Region Server)

       4: No of Container Launched are same in both case

      HBase 2 took 10% more time then HBase 1.3 where test input is same for both cluster

       

      Feature HBase 2.2.3
      Time(Sec)      		 HBase 1.3.1
      Time(Sec)      		 Diff% Snappy lib:
       
      BulkLoad 21837 19686.16 -10.93 Snappy lib:
      HBase 2.2.3: 1.4
      HBase 1.3.1: 1.4

      Attachments

        1. Branch1.3_putSortReducer_sampleCode.patch
          5 kB
          Y. SREENIVASULU REDDY
        2. Branch2_putSortReducer_sampleCode.patch
          5 kB
          Y. SREENIVASULU REDDY
        3. Branc2_withComparator_atKeyValue.patch
          10 kB
          ramkrishna.s.vasudevan
        4. flamegraph_branch-1_new.svg
          14 kB
          ramkrishna.s.vasudevan
        5. flamegraph_branch-2_afterpatch.svg
          11 kB
          ramkrishna.s.vasudevan
        6. flamegraph_branch-2.svg
          24 kB
          ramkrishna.s.vasudevan

        Issue Links

          is fixed by

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-24850 CellComparator perf improvement

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              a00408367 Ajeet Rai
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated: