Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-7506

[Java] JMH benchmarks should be called from main methods

    XMLWordPrintableJSON

Details

    Description

      Some benchmarks are called as unit tests in our current code base. They should be called from main methods, because:

      1. This is the recommended way of writing JMH benchmarks. The automatically generated benchmarks are called from main, and sample benchmarks provided by JMH [1] are also called from main.

      2. Some compiler does not support calling JMH as unit test. For example, the "javac with error prone" reports the following error:

      Error:(100, 15) java: [JUnit4TearDownNotRun] tearDown() method will not be run; please add JUnit's @After annotation
      (see https://errorprone.info/bugpattern/JUnit4TearDownNotRun)
      Did you mean '@After'?

      3. When run as a unit test, enable assert flag will be turned on by default, so some test/debug operations will be performed. This will distort the benchmark result data. For example, a related discussion can be found in [2].

      [1] https://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/
      [2] https://github.com/apache/arrow/pull/5842#issuecomment-558082914

      Attachments

        Issue Links

          Activity

            People

              fan_li_ya Liya Fan
              fan_li_ya Liya Fan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h