Uploaded image for project: 'Commons Math'
  1. Commons Math
  2. MATH-1257

NormalDistribution.cumulativeProbability() suffers from cancellation

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 3.5
    • 4.0, 3.6
    • None
    • None

    Description

      I see the following around line 194:

              return 0.5 * (1 + Erf.erf(dev / (standardDeviation * SQRT2)));
      

      When erf() returns a very small value, this cancels in the addition with the "1.0" which leads to poor precision in the results.

      I would suggest changing this line to read more like:

      return 0.5 * Erf.erfc( -dev / standardDeviation * SQRT2 );
      

      Should you want some test cases for "extreme values" (one might argue that within 10 standard deviations isn't all that extreme) then you can check the following: http://www.jstatsoft.org/v52/i07/ then look in the v52i07-xls.zip at replication-01-distribution-standard-normal.xls

      I think you will also find that evaluation of expressions such as

      NormalDistribution( 0, 1 ).cumulativeProbability( -10.0 );

      are pretty far off.

      Attachments

        1. MATH-1257.patch
          2 kB
          Gilles Sadowski

        Activity

          People

            Unassigned Unassigned
            murphyw Bill Murphy
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: