[AVRO-1864] Decimal conversion should convert values with scale less than intended scale instead of erroring.

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 1.7.6
• Assignee: Ryan Blue
• Reporter: Thomas Scott
• Created: 16/Jun/16 10:16
• Updated: 20/Jun/16 17:25
• Resolved: 20/Jun/16 17:25

Description

Using Sqoop to import data to AVRO can mean that decimal scales in the incoming values do not match the scales expected in AVRO. In this situation AVRO file creation fails. However, in some cases this is not the correct behaviour, given a value of 3.1 and a scale of 3 the value will fit into the scale requirements and so should be adjusted.

Looking through the code this seems to be enforced here:

src/main/java/org/apache/avro/Conversions.java
public ByteBuffer toBytes(BigDecimal value, Schema schema, LogicalType type) {
int scale = ((LogicalTypes.Decimal) type).getScale();
if (scale != value.scale()) {
throw new AvroTypeException("Cannot encode decimal with scale " +
value.scale() + " as scale " + scale);

Should this not be:

if (scale < value.scale()) {

The same applies in: toFixed()