Uploaded image for project: 'Log4j 2'
  1. Log4j 2
  2. LOG4J2-2478

JMH Benchmarks in not consuming all computed variables (risk of DCE)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Tests
    • Labels:
      None
    • Environment:

      I am reporting the result of 5 full executions with default parameters (iterations, forks, warmups, etc..)

      Tests run on a Computational server with CPU: E5-1660-3.3GHZ (6 cores + HT), 64 GB RAM.

      Description

      We are conducting a scientific study to investigate bad practices/anti-patterns on creating micro-benchmarks using JMH, and we found an instance of harmfull non-consumed computation in `AbstractStringLayoutStringEncodingBenchmark`.

      As a good practice, every computation performed in inside the benchmark should be consumed (see [JMH Documentation|http://hg.openjdk.java.net/code-tools/jmh/file/66fb723292d4/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_08_DeadCode.java.] 

      This is partially done here with the self-made consume method, but on some benchmarks (such as baseline, usAsciiGetBytes) the return is not consumed in the benchmark, which opens the possibility of a Dead-Code Elimination to be performed by JVM. 

      There is a very simple solution: just return the long primitive calculated by consume() at the end of each benchmark. This is done on other benchmarks of the project as well.

      In our tests, benchmarks were lightly affected (see attachment), with Throughputs 5-35% lower after our fix patch. 

       

        Attachments

        1. logginglog4j2-jmhantipatterns-retu.csv
          1 kB
          Diego Elias Damasceno Costa

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                DiegoEliasCosta Diego Elias Damasceno Costa
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: