[MATH-1067] Stack overflow in Beta.regularizedBeta

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2
• Fix Version/s: 3.3
• Labels: easyfix performance

Environment

Java build 1.7.0_45-b18

• Assignee: Unassigned
• Reporter: Florian Erhard
• Created: 28/Nov/13 10:47
• Updated: 19/May/14 15:13
• Resolved: 28/Nov/13 11:42

Description

In org.apache.commons.math3.special.Beta.regularizedBeta(double,double,double,double,int), the case

} else if (x > (a + 1.0) / (a + b + 2.0)) {
ret = 1.0 - regularizedBeta(1.0 - x, b, a, epsilon, maxIterations);
}

is prone to infinite recursion: If x is approximately the tested value, then 1-x is approximately the tested value in the recursion. Thus, due to loss of precision after the subtraction, this condition can be true for the recursive call as well.

Example:
double x= Double.longBitsToDouble(4597303555101269224L);
double a= Double.longBitsToDouble(4634227472812299606L);
double b = Double.longBitsToDouble(4642050131540049920L);
System.out.println(x > (a + 1.0) / (a + b + 2.0));
System.out.println(1-x>(b + 1.0) / (b + a + 2.0));
System.out.println(1-(1-x)>(a + 1.0) / (a + b + 2.0));

Possible solution: change the condition to
x > (a + 1.0) / (a + b + 2.0) && 1-x<=(b + 1.0) / (b + a + 2.0)