[GEODE-9641] Benchmark instability in P2pPartitionedPutBytesBenchmark

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: membership
• Labels: pull-request-available testing
• Assignee: Kamilla Aslami
• Reporter: Kamilla Aslami
• Created: 25/Sep/21 06:25
• Updated: 22/Jun/22 20:46
• Resolved: 12/Oct/21 12:53

Description

P2pPartitionedPutBytesBenchmark started failing in CI after the fix for GEODE-9204 was merged.

org.apache.geode.benchmark.tests.P2pPartitionedPutBytesBenchmark
              average ops/second  Baseline:    179582.83  Test:    170268.31  Difference:   -5.2%
       ops/second standard error  Baseline:       849.15  Test:       649.44  Difference:  -23.5%
   ops/second standard deviation  Baseline:     25346.86  Test:     19374.76  Difference:  -23.6%
      YS 99th percentile latency  Baseline:      5010.00  Test:      5110.00  Difference:   +2.0%
                  median latency  Baseline:   2416639.00  Test:   2412543.00  Difference:   -0.2%
         90th percentile latency  Baseline:   4698111.00  Test:   4567039.00  Difference:   -2.8%
         99th percentile latency  Baseline:  38141951.00  Test:  81723391.00  Difference: +114.3%
       99.9th percentile latency  Baseline: 196214783.00  Test: 205389823.00  Difference:   +4.7%
                 average latency  Baseline:   4033453.68  Test:   4258029.39  Difference:   +5.6%
      latency standard deviation  Baseline:  14549153.77  Test:  15673393.37  Difference:   +7.7%
          latency standard error  Baseline:      1149.00  Test:      1271.79  Difference:  +10.7%
              average ops/second  Baseline:    533863.27  Test:    505986.13  Difference:   -5.2%
BENCHMARK FAILED: org.apache.geode.benchmark.tests.P2pPartitionedPutBytesBenchmark average latency is 5% worse than baseline.

Issue Links

• links to: GitHub Pull Request #6943