Uploaded image for project: 'Commons Math'
  1. Commons Math
  2. MATH-1154

Statistical tests in stat.inference package are very slow due to implicit RandomGenerator initialization

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3
    • Fix Version/s: 3.4
    • Labels:
      None

      Description

      Some statistical tests defined in the stat.inference package (e.g. BinomialTest or ChiSquareTest) are unnecessarily very slow (up to a factor 20 slower than necessary). The reason is the implicit slow initialization of a default (Well19937c) random generator instance each time a test is performed. The affected tests create some distribution instance in order to use some methods defined therein. However, they do not use any method for random generation. Nevertheless a random number generator instance is automatically created when creating a distribution instance, which is the reason for the serious slowdown. The problem is related to MATH-1124.

      There are following solutions:
      1) Fix the affected statistical tests by passing a light-weight RandomGenerator implementation (or even null) to the constructor of the distribution.
      2) Or use for all distributions a RandomGenerator implementation that uses lazy initialization to generate the Well19937c instance as late as possible. This would also solve MATH-1124.

      I will attach a patch proposal together with a performance test, that will demonstrate the speed up after a fix.

        Attachments

        1. math3.patch
          35 kB
          Otmar Ertl
        2. MATH-1154.patch
          10 kB
          Thomas Neidhart

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                Otmar Ertl Otmar Ertl
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: