[ARROW-1716] [Format/JSON] Use string integer value for Decimals in JSON

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.7.1
• Fix Version/s: 0.8.0
• Component/s: C++, Java
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/17723
• Assignee: Phillip Cloud
• Reporter: Phillip Cloud
• Created: 23/Oct/17 22:36
• Updated: 11/Jan/23 07:16
• Resolved: 07/Nov/17 21:22

Description

Suprisingly, Java and C++ integration tests pass after ARROW-1588. This hides a bug, because we're writing decimal values as hex encoded bytes.

C++ and Java compare that the bytes are the same, but because C++ is interpreting everything as little endian after ARROW-1588 and Java is big endian the numbers these bytes represent will be different in their respective systems.

I propose that instead of encoding DecimaArray/DecimalVector values as hex encoded bytes, we store the integer as a string when writing Arrow DecimalArray/DecimalVector data to JSON. This will allow us to compare that the bytes have the same meaning in both systems.

This requires a change to the way Arrow writes JSON.

icexelloss was extremely helpful in helping me get to the bottom of this.

cc icexelloss wesmckinn jnadeau

Issue Links

• links to: GitHub Pull Request #1267