Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
1.36.0
Description
Consider this test that could be a SqlOperatorTest:
f.checkScalar("CAST(CAST('36854775807.0' AS REAL) AS BIGINT)", "36854775808", "BIGINT NOT NULL");
The produced result is actually very far:
Expected: is "36854775808"
but: was "36854779904"
This big error comes from two reasons:
- Calcite uses BigDecimal values to represent floating point values, see
CALCITE-2067 - When converting a Float value to a BigDecimal in RexBuilder.clean(), the following sequence is used:
new BigDecimal(((Number) o).doubleValue(), MathContext.DECIMAL32)
Using a DECIMAL32 math context leads to the precision loss. Just because a Float uses 32 bits does not mean that the decimal should also use 32 bits.
The real fix is in the PR for CALCITE-2067, but that hasn't been reviewed for a long time, so I will submit a fix for the clean() method..
Attachments
Issue Links
- links to