[HBASE-15405] Synchronize final results logging single thread in PE, fix wrong defaults in help message

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.3.0, 0.98.19, 2.0.0
• Component/s: Performance
• Hadoop Flags: Reviewed
• Assignee: Apekshit Sharma
• Reporter: Apekshit Sharma
• Created: 06/Mar/16 07:53
• Updated: 09/Nov/17 00:41
• Resolved: 14/Apr/16 16:01

Description

Corrects wrong default values for few options in the help message.

Final stats from multiple clients are intermingled making it hard to understand. Also the logged stats aren't very machine readable. It can be helpful in a daily perf testing rig which scraps logs for results.

Example of logs before the change.

16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest latency log (microseconds), on 1048576 measures
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest latency log (microseconds), on 1048576 measures
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest latency log (microseconds), on 1048576 measures
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest latency log (microseconds), on 1048576 measures
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest latency log (microseconds), on 1048576 measures
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest latency log (microseconds), on 1048576 measures
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest latency log (microseconds), on 1048576 measures
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: 0/1048570/1048576, latency mean=953.98, min=359.00, max=324050.00, stdDev=851.82, 95th=1368.00, 99th=1625.00
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: 0/1048570/1048576, latency mean=953.92, min=356.00, max=323394.00, stdDev=817.55, 95th=1370.00, 99th=1618.00
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: 0/1048570/1048576, latency mean=953.98, min=367.00, max=322745.00, stdDev=840.43, 95th=1369.00, 99th=1622.00
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest latency log (microseconds), on 1048576 measures
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest latency log (microseconds), on 1048576 measures
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest latency log (microseconds), on 1048576 measures
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest Min      = 375.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest Min      = 363.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest Avg      = 953.6624126434326
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest Avg      = 953.4124526977539
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest StdDev   = 781.3929776087633
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest StdDev   = 742.8027916717297
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 50th     = 894.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 50th     = 894.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 75th     = 1070.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 75th     = 1071.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 95th     = 1369.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 95th     = 1369.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 99th     = 1623.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 99th     = 1624.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest Min      = 372.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 99.9th   = 3013.9980000000214
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest Avg      = 953.2451229095459
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 99.9th   = 3043.9980000000214
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest StdDev   = 725.4744472152282
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 99.99th  = 25282.380199996755
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 50th     = 895.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 99.99th  = 25812.763399994
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 75th     = 1071.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 99.999th = 89772.78990004538
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 95th     = 1369.0
16/03/05 22:43:06 INFO hbase.PerformanceEvaluation: IncrementTest 99.999th = 122808.39587019826

After the change

16/03/06 02:38:21 INFO hbase.PerformanceEvaluation: Test : RandomWriteTest, Thread : TestClient-1
16/03/06 02:38:21 INFO hbase.PerformanceEvaluation: Latency (us) : , mean=6.23, min=2.00, max=101433.00, stdDev=246.62, 50th=2.00, 75th=2.00, 95th=3.00, 99th=13.00, 99.9th=558.00, 99.99th=9656.19, 99.999th=20213.63
16/03/06 02:38:21 INFO hbase.PerformanceEvaluation: Num measures (latency) : 1048576
16/03/06 02:38:21 INFO hbase.PerformanceEvaluation: ValueSize (bytes) : , mean=0.00, min=0.00, max=0.00, stdDev=0.00, 50th=0.00, 75th=0.00, 95th=0.00, 99th=0.00, 99.9th=0.00, 99.99th=0.00, 99.999th=0.00
16/03/06 02:38:21 INFO hbase.PerformanceEvaluation: Num measures (ValueSize): 0
...
16/03/06 02:38:21 INFO hbase.PerformanceEvaluation: Test : RandomWriteTest, Thread : TestClient-0
16/03/06 02:38:21 INFO hbase.PerformanceEvaluation: Latency (us) : , mean=6.23, min=1.00, max=99321.00, stdDev=246.41, 50th=2.00, 75th=2.00, 95th=3.00, 99th=13.00, 99.9th=566.00, 99.99th=9694.60, 99.999th=23311.86
....