Uploaded image for project: 'Log4j 2'
  1. Log4j 2
  2. LOG4J2-2478

JMH Benchmarks in not consuming all computed variables (risk of DCE)

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • None
    • Tests
    • None
    • I am reporting the result of 5 full executions with default parameters (iterations, forks, warmups, etc..)

      Tests run on a Computational server with CPU: E5-1660-3.3GHZ (6 cores + HT), 64 GB RAM.

    Description

      We are conducting a scientific study to investigate bad practices/anti-patterns on creating micro-benchmarks using JMH, and we found an instance of harmfull non-consumed computation in `AbstractStringLayoutStringEncodingBenchmark`.

      As a good practice, every computation performed in inside the benchmark should be consumed (see [JMH Documentation|http://hg.openjdk.java.net/code-tools/jmh/file/66fb723292d4/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_08_DeadCode.java.] 

      This is partially done here with the self-made consume method, but on some benchmarks (such as baseline, usAsciiGetBytes) the return is not consumed in the benchmark, which opens the possibility of a Dead-Code Elimination to be performed by JVM. 

      There is a very simple solution: just return the long primitive calculated by consume() at the end of each benchmark. This is done on other benchmarks of the project as well.

      In our tests, benchmarks were lightly affected (see attachment), with Throughputs 5-35% lower after our fix patch. 

       

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            DiegoEliasCosta Diego Elias Damasceno Costa
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment