[MATH-588] Weighted Mean evaluation may not have optimal numerics

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1, 2.2
• Fix Version/s: 3.0
• Assignee: Unassigned
• Reporter: Phil Steitz
• Created: 12/Jun/11 18:19
• Updated: 24/Mar/12 16:16
• Resolved: 05/Feb/12 19:54

Description

I recently got this in a test run

testWeightedConsistency(org.apache.commons.math.stat.descriptive.moment.MeanTest)  Time elapsed: 0 sec  <<< FAILURE!
java.lang.AssertionError: expected:<0.002282165958997601> but was:<0.002282165958997157>
	at org.junit.Assert.fail(Assert.java:91)
	at org.junit.Assert.failNotEquals(Assert.java:645)
	at org.junit.Assert.assertEquals(Assert.java:441)
	at org.apache.commons.math.TestUtils.assertRelativelyEquals(TestUtils.java:178)
	at org.apache.commons.math.TestUtils.assertRelativelyEquals(TestUtils.java:153)
	at org.apache.commons.math.stat.descriptive.UnivariateStatisticAbstractTest.testWeightedConsistency(UnivariateStatisticAbstractTest.java:170)

The correction formula used to compute the unweighted mean may not be appropriate or optimal in the presence of weights:

// Compute initial estimate using definitional formula
double sumw = sum.evaluate(weights,begin,length);
double xbarw = sum.evaluate(values, weights, begin, length) / sumw;

// Compute correction factor in second pass
double correction = 0;
for (int i = begin; i < begin + length; i++) {
  correction += weights[i] * (values[i] - xbarw);
}
return xbarw + (correction/sumw);