Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
1.8.2, 1.9.2
-
None
Description
Came across an interesting issue in Avro 1.8.2
Configured a decimal logical type (Fixed type of size 12 with scale of 15 and precision of 28).
Due to an upstream bug, a value of 1070464558597365250.000000000000000 (1.07046455859736525E+18 that is then rescaled to 15) appears, and the DecimalConversion attempts to turn it into a Fixed type.
This should have failed, as it has a precision of 34 and won't fit into the 12 bytes (needs 14). However in 1.8.2 this still writes a value that downstream processing then works out is invalid and errors.
Basically the top 2 bytes are thrown away.
This problem is fixed in 1.9.2 due to the change in https://issues.apache.org/jira/browse/AVRO-2309 as this particular issue fails when it attempts to pass an offset of -2 to the System.arraycopy method.
That seems ok, but is a bit of a red herring to the actual issue, and precision is still not actually being checked.
Proposing a couple changes to the DecimalConversion:
- Check precision set on the decimal logical type. If value has greater precision then error with more informative message
- Still check scale and error if value has a greater scale. However if the scale in value is less, than it seems safe to update the scale and pad zero's rather than error
- Do this for both Bytes and Fixed types