[DRILL-3429] DrillAvgVarianceConvertlet may produce wrong results while rewriting stddev, variance

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Assignee: Mehant Baid
• Reporter: Mehant Baid
• Reviewer: Victoria Markman
• Created: 30/Jun/15 18:04
• Updated: 17/Dec/15 18:32
• Resolved: 23/Oct/15 06:51

Description

DrillAvgVarianceConvertlet currently rewrites aggregate functions like avg, stddev, variance to simple computations.

Eg:
Stddev( x ) => power(
(sum(x * x) - sum( x ) * sum( x ) / count( x ))
/ count( x ),
.5)

Consider the case when the input is an integer. Now the rewrite contains multiplication and division, which will bind to functions that operate on integers however the expected result should be a double and since double has more precision than integer we should be operating on double during the multiplication and division.