[PIG-5328] expressionOperator Divide.equalsZero(DataType.BIGDECIMAL) is invalid

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.16.0, 0.17.0
• Fix Version/s: 0.18.0
• Component/s: impl

Environment

pig source HEAD as of Jan 2018 ... probably goes all the way back to initial implementation of BigDecimal support

• Hadoop Flags: Reviewed

Release Note

the division operator checks for a zero denominator to prevent a divide-by-zero exception. Unlike other numeric types, BigDecimal has multiple representations of zero with different scales ... 0, 0.0, 0.00, etc. Previous versions of pig were were not properly detecting zero values with non-zero scales when performing denominator check prior to division.

• Assignee: Michael Howard
• Reporter: Michael Howard
• Created: 17/Jan/18 19:21
• Updated: 27/Jan/18 04:18
• Resolved: 27/Jan/18 04:18

Description

Divide.equalsZero(DataType.BIGDECIMAL) is flawed in that it uses an invalid test for == ZERO in the case of BigDecimal. 

 

./physicalLayer/expressionOperators/Divide.java tests the divisor for zero in order to avoid DivideByZero.

The test is performed using a method equalsZero(...)

Divide.equalsZero() is given 'protected' access, but I could not find other references ... should be 'private'

equalsZero() implementation dispatches on dataType to type-specific predicates ... the BigDecimal implementation is incorrect 

The method BigDecimal.equals(other) is intended to be used for object equality, not numerical equality. (Their justification is that equals() is used in hash-table lookups in java Collections.) BigDecimal numbers are not normalized and scale is an important attribute. Scale is included in BigDecimal.equals(). The values "0" and "0.00" have different scales and are not considered "equals()"

Comparisons for numeric equality need to be done using compareTo()

In the special case of comparing to zero, BigDecimal.signum() is the best. 

The current code is

     case DataType.BIGDECIMAL:

            return BigDecimal.ZERO.equals((BigDecimal) a);

needs to be changed to

     case DataType.BIGDECIMAL:

            return ((BigDecimal) a).signum() == 0;