[LOG4J2-2478] JMH Benchmarks in not consuming all computed variables (risk of DCE)

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Component/s: Tests

Environment

I am reporting the result of 5 full executions with default parameters (iterations, forks, warmups, etc..)

Tests run on a Computational server with CPU: E5-1660-3.3GHZ (6 cores + HT), 64 GB RAM.

• Assignee: Unassigned
• Reporter: Diego Elias Damasceno Costa
• Created: 15/Oct/18 11:20
• Updated: 17/Oct/18 00:08
• Resolved: 16/Oct/18 23:58

Description

We are conducting a scientific study to investigate bad practices/anti-patterns on creating micro-benchmarks using JMH, and we found an instance of harmfull non-consumed computation in `AbstractStringLayoutStringEncodingBenchmark`.

As a good practice, every computation performed in inside the benchmark should be consumed (see [JMH Documentation|http://hg.openjdk.java.net/code-tools/jmh/file/66fb723292d4/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_08_DeadCode.java.] 

This is partially done here with the self-made consume method, but on some benchmarks (such as baseline, usAsciiGetBytes) the return is not consumed in the benchmark, which opens the possibility of a Dead-Code Elimination to be performed by JVM. 

There is a very simple solution: just return the long primitive calculated by consume() at the end of each benchmark. This is done on other benchmarks of the project as well.

In our tests, benchmarks were lightly affected (see attachment), with Throughputs 5-35% lower after our fix patch. 

 

Issue Links

• links to: GitHub Pull Request #219