# hbase pe performace test hbase 2.0.2(hdp 3.1.1) ## randomWrite [hadoop@bigdata92136 ~]$ hbase pe --nomapred --oneCon=true --valueSize=100 --compress=SNAPPY --rows=150000 --autoFlush=true --presplit=24 randomWrite 24 2020-12-01 18:01:24,477 INFO [TestClient-14] hbase.PerformanceEvaluation: Finished class org.apache.hadoop.hbase.PerformanceEvaluation$RandomWriteTest in 175677ms at offset 2100000 for 150000 rows (0.11 MB/s) 2020-12-01 18:01:24,477 INFO [TestClient-14] hbase.PerformanceEvaluation: Finished TestClient-14 in 175677ms over 150000 rows 2020-12-01 18:01:24,866 INFO [TestClient-5] hbase.PerformanceEvaluation: Test : RandomWriteTest, Thread : TestClient-5 2020-12-01 18:01:24,879 INFO [TestClient-5] hbase.PerformanceEvaluation: Latency (us) : mean=1171.77, min=497.00, max=558309.00, stdDev=3019.29, 50th=1054.00, 75th=1174.00, 95th=1370.00, 99th=2058.00, 99.9th=21760.00, 99.99th=133672.00, 99.999th=525393.50 2020-12-01 18:01:24,879 INFO [TestClient-5] hbase.PerformanceEvaluation: Num measures (latency) : 149999 2020-12-01 18:01:24,892 INFO [TestClient-5] hbase.PerformanceEvaluation: Mean = 1171.77 Min = 497.00 Max = 558309.00 StdDev = 3019.29 50th = 1054.00 75th = 1174.00 95th = 1370.00 99th = 2058.00 99.9th = 21760.00 99.99th = 133672.00 99.999th = 525393.50 2020-12-01 18:01:24,893 INFO [main] hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [174122, 174677, 174701, 174583, 174459, 176067, 174303, 174748, 174754, 175097, 174132, 174974, 175098, 173743, 175677, 175588, 173958, 174019, 174335, 175126, 173828, 174751, 175354, 174633] 2020-12-01 18:01:24,894 INFO [main] hbase.PerformanceEvaluation: [RandomWriteTest] Min: 173743ms Max: 176067ms Avg: 174696ms ## sequentialWrite ### step1. perpare data [hadoop@bigdata92136 ~]$ hbase pe --nomapred --oneCon=true --valueSize=100 --compress=SNAPPY --size=2 --presplit=24 sequentialWrite 24 2020-12-01 18:10:29,934 INFO [TestClient-1] hbase.PerformanceEvaluation: Finished class org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest in 370604ms at offset 873813 for 873813 rows (0.3 MB/s) 2020-12-01 18:10:29,934 INFO [TestClient-1] hbase.PerformanceEvaluation: Finished TestClient-1 in 370604ms over 873813 rows 2020-12-01 18:10:34,805 INFO [TestClient-12] hbase.PerformanceEvaluation: 10485756/11324556/11359569, latency mean=409.68, min=0.00, max=1906928.00, stdDev=22400.72, 95th=1.00, 99th=12.00 2020-12-01 18:10:37,153 INFO [TestClient-12] hbase.PerformanceEvaluation: Test : SequentialWriteTest, Thread : TestClient-12 2020-12-01 18:10:37,168 INFO [TestClient-12] hbase.PerformanceEvaluation: Latency (us) : mean=400.22, min=0.00, max=1906928.00, stdDev=22108.03, 50th=1.00, 75th=1.00, 95th=1.00, 99th=11.00, 99.9th=32.00, 99.99th=1413080.02, 99.999th=1514906.56 2020-12-01 18:10:37,168 INFO [TestClient-12] hbase.PerformanceEvaluation: Num measures (latency) : 873812 2020-12-01 18:10:37,181 INFO [TestClient-12] hbase.PerformanceEvaluation: Mean = 400.22 Min = 0.00 Max = 1906928.00 StdDev = 22108.03 50th = 1.00 75th = 1.00 95th = 1.00 99th = 11.00 99.9th = 32.00 99.99th = 1413080.02 99.999th = 1514906.56 2020-12-01 18:10:37,182 INFO [TestClient-12] hbase.PerformanceEvaluation: Finished class org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest in 377844ms at offset 10485756 for 873813 rows (0.29 MB/s) 2020-12-01 18:10:37,182 INFO [TestClient-12] hbase.PerformanceEvaluation: Finished TestClient-12 in 377844ms over 873813 rows 2020-12-01 18:10:37,183 INFO [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Summary of timings (ms): [323286, 370604, 367497, 333265, 293601, 150600, 364796, 340508, 366935, 367884, 286218, 300543, 377844, 323782, 317694, 281854, 253457, 331318, 344839, 293201, 333873, 342135, 288716, 283478] 2020-12-01 18:10:37,185 INFO [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Min: 150600ms Max: 377844ms Avg: 318247ms ### step2. random read [hadoop@bigdata92136 ~]$ hbase pe --nomapred --oneCon=true --valueSize=100 --size=2 randomRead 100 2020-12-01 18:38:25,692 INFO [TestClient-73] hbase.PerformanceEvaluation: Finished class org.apache.hadoop.hbase.PerformanceEvaluation$RandomReadTest in 1657397ms at offset 15309195 for 209715 rows (0.02 MB/s) 2020-12-01 18:38:25,692 INFO [TestClient-73] hbase.PerformanceEvaluation: Finished TestClient-73 in 1657397ms over 209715 rows 2020-12-01 18:38:26,307 INFO [TestClient-69] hbase.PerformanceEvaluation: 14470335/14679700/14680050, latency mean=7915.83, min=321.00, max=694161.00, stdDev=48610.18, 95th=4639.00, 99th=311021.12 2020-12-01 18:38:26,761 INFO [TestClient-69] hbase.PerformanceEvaluation: Test : RandomReadTest, Thread : TestClient-69 2020-12-01 18:38:26,778 INFO [TestClient-69] hbase.PerformanceEvaluation: Latency (us) : mean=7904.82, min=321.00, max=694161.00, stdDev=48570.51, 50th=868.00, 75th=1020.00, 95th=4634.00, 99th=310126.90, 99.9th=537230.80, 99.99th=605172.74, 99.999th=677274.13 2020-12-01 18:38:26,778 INFO [TestClient-69] hbase.PerformanceEvaluation: Num measures (latency) : 209714 2020-12-01 18:38:26,794 INFO [TestClient-69] hbase.PerformanceEvaluation: Mean = 7904.82 Min = 321.00 Max = 694161.00 StdDev = 48570.51 50th = 868.00 75th = 1020.00 95th = 4634.00 99th = 310126.90 99.9th = 537230.80 99.99th = 605172.74 99.999th = 677274.13 2020-12-01 18:38:26,795 INFO [TestClient-69] hbase.PerformanceEvaluation: Finished class org.apache.hadoop.hbase.PerformanceEvaluation$RandomReadTest in 1658435ms at offset 14470335 for 209715 rows (0.02 MB/s) 2020-12-01 18:38:26,795 INFO [TestClient-69] hbase.PerformanceEvaluation: Finished TestClient-69 in 1658435ms over 209715 rows 2020-12-01 18:38:26,795 INFO [main] hbase.PerformanceEvaluation: [RandomReadTest] Summary of timings (ms): [1636454, 1654108, 1645900, 1634876, 1643609, 1644172, 1652218, 1642298, 1655427, 1636089, 1641424, 1652246, 1642842, 1627480, 1655656, 1644990, 1649646, 1655638, 1654585, 1635725, 1653166, 1653823, 1652740, 1654094, 1638494, 1654510, 1648915, 1654074, 1647005, 1649639, 1653550, 1643683, 1654376, 1639164, 1654294, 1646645, 1656229, 1655845, 1627820, 1650355, 1651712, 1656529, 1619753, 1648281, 1652250, 1656665, 1655624, 1632669, 1649319, 1653115, 1650759, 1630715, 1654404, 1613872, 1644034, 1654338, 1651447, 1633662, 1653692, 1647837, 1645718, 1637185, 1654646, 1627989, 1646564, 1605954, 1652154, 1633165, 1653453, 1658435, 1649223, 1657367, 1655111, 1657397, 1651757, 1634820, 1652021, 1652752, 1648895, 1643392, 1654913, 1652694, 1656067, 1641137, 1652649, 1656632, 1651841, 1630879, 1652311, 1654521, 1651232, 1639386, 1655191, 1655368, 1657129, 1649734, 1623735, 1638623, 1632124, 1651799] 2020-12-01 1