[GEODE-9778] Benchmark degradation in RedisHsetBenchmark since netty bump

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.14.1, 1.15.0
• Component/s: benchmarks, redis
• Labels: pull-request-available testing unreleased
• Assignee: Jens Deppe
• Reporter: Owen Nichols
• Created: 27/Oct/21 06:11
• Updated: 08/Feb/22 16:30
• Resolved: 05/Nov/21 19:48

Description

examples:

https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/benchmark-radish/builds/179

#179 seems to show a consistent 10-15% degradation in only in the Hset benchmark

...
This is ITERATION 3 of benchmarking against baseline.
                                          RedisHsetBenchmark avg ops/sec  Baseline:    234210.78  Test:    207591.61  Difference:  -11.4%
                                                             avg latency  Baseline:   3070166.30  Test:   3468289.49  Difference:  +13.0%
This is ITERATION 4 of benchmarking against baseline.
                                          RedisHsetBenchmark avg ops/sec  Baseline:    238884.30  Test:    216618.91  Difference:   -9.3%
                                                             avg latency  Baseline:   3016750.52  Test:   3319615.79  Difference:  +10.0%
This is ITERATION 5 of benchmarking against baseline.
                                          RedisHsetBenchmark avg ops/sec  Baseline:    248656.29  Test:    211220.74  Difference:  -15.1%
                                                             avg latency  Baseline:   2894368.00  Test:   3405200.04  Difference:  +17.6% 

 

Also https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/benchmark-radish/builds/180 shows similar results:

This is ITERATION 2 of benchmarking against baseline.
                                   RedisHsetAndHgetBenchmark avg ops/sec  Baseline:    146409.80  Test:    134019.47  Difference:   -8.5%
                                                             avg latency  Baseline:   4914049.28  Test:   5365514.18  Difference:   +9.2%
                                          RedisHsetBenchmark avg ops/sec  Baseline:    217729.26  Test:    191512.40  Difference:  -12.0%
                                                             avg latency  Baseline:   3302672.13  Test:   3750731.05  Difference:  +13.6%
This is ITERATION 3 of benchmarking against baseline.
                                   RedisHsetAndHgetBenchmark avg ops/sec  Baseline:    141960.20  Test:    129616.40  Difference:   -8.7%
                                                             avg latency  Baseline:   5065570.96  Test:   5554949.31  Difference:   +9.7%
                                          RedisHsetBenchmark avg ops/sec  Baseline:    219985.85  Test:    199572.01  Difference:   -9.3%
                                                             avg latency  Baseline:   3268138.34  Test:   3603197.51  Difference:  +10.3%
This is ITERATION 4 of benchmarking against baseline.
                                   RedisHsetAndHgetBenchmark avg ops/sec  Baseline:    140404.88  Test:    134341.05  Difference:   -4.3%
                                                             avg latency  Baseline:   5127629.59  Test:   5354599.94  Difference:   +4.4%
                                          RedisHsetBenchmark avg ops/sec  Baseline:    224728.82  Test:    194024.96  Difference:  -13.7%
                                                             avg latency  Baseline:   3205365.68  Test:   3706058.86  Difference:  +15.6%
This is ITERATION 5 of benchmarking against baseline.
                                          RedisHsetBenchmark avg ops/sec  Baseline:    220019.47  Test:    192107.91  Difference:  -12.7%
                                                             avg latency  Baseline:   3268273.76  Test:   3744842.51  Difference:  +14.6%
 

 

Issue Links

• is duplicated by: GEODE-9779 Redis benchmarks are failing in RedisHsetBenchmark
• links to: GitHub Pull Request #7075
• links to: GitHub Pull Request #7084