[KAFKA-9667] Connect JSON serde strip trailing zeros

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.6.0, 2.4.2, 2.5.1
• Component/s: connect
• Assignee: Andy Coates
• Reporter: Andy Coates
• Reviewer: Randall Hauch
• Created: 05/Mar/20 17:32
• Updated: 07/May/20 22:43
• Resolved: 07/May/20 22:43

Description

The Connect Json serde was recently enhanced to support serializing decimals as standard JSON numbers, (Original work done under KAFKA-8595), e.g. `1.23`.  However, there is a bug in the implementation: it's stripping trailing zeros!  `1.23` is not the same as `1.230`.  The first is a number accurate to 2 decimal places, where as the later is accurate to 3 dp.

It is important that trailing zeros are not dropped when de(serializing) decimals.  For some use-cases it may be acceptable to drop the trailing zeros, but for others it definitely is not.

Current Functionality

If a JSON object was to contain the number `1.230` then the Java JsonDeserializer would correctly deserialize this into a `BigDecimal`. The BigDecimal would have a scale of 3, which is correct.

However, if that same BigDecimal was then serialized back to JSON using the Java JsonSerializer it would incorrectly strip the zeros, serializing to `1.23`. 

Expected Functionality

When serializing, trailing zeros should be maintained.  For example, a BigDecimal such as `1.230`, (3 dp), should be serialized as `1.230`. 

Compatibility

With this bug fix any computation on decimal values that resulted in a number with trailing zeros, would see the number correctly serialized with those trailing zeros.

Both the old serialized number, e.g. `1.23`, and the proposed corrected serialized number, e.g. `1.230`, are valid JSON numbers. Downstream consumers should have no issue deserializing either.  

 

 

Issue Links

• links to: GitHub Pull Request #8230