Uploaded image for project: 'Commons Math'
  1. Commons Math
  2. MATH-1154

Statistical tests in stat.inference package are very slow due to implicit RandomGenerator initialization

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.3
    • 3.4
    • None
    • None

    Description

      Some statistical tests defined in the stat.inference package (e.g. BinomialTest or ChiSquareTest) are unnecessarily very slow (up to a factor 20 slower than necessary). The reason is the implicit slow initialization of a default (Well19937c) random generator instance each time a test is performed. The affected tests create some distribution instance in order to use some methods defined therein. However, they do not use any method for random generation. Nevertheless a random number generator instance is automatically created when creating a distribution instance, which is the reason for the serious slowdown. The problem is related to MATH-1124.

      There are following solutions:
      1) Fix the affected statistical tests by passing a light-weight RandomGenerator implementation (or even null) to the constructor of the distribution.
      2) Or use for all distributions a RandomGenerator implementation that uses lazy initialization to generate the Well19937c instance as late as possible. This would also solve MATH-1124.

      I will attach a patch proposal together with a performance test, that will demonstrate the speed up after a fix.

      Attachments

        1. math3.patch
          35 kB
          Otmar Ertl
        2. MATH-1154.patch
          10 kB
          Thomas Neidhart

        Issue Links

          Activity

            People

              Unassigned Unassigned
              Otmar Ertl Otmar Ertl
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: