[THRIFT-1727] Ruby-1.9: data loss: "binary" fields are re-encoded

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9
• Fix Version/s: 0.9.2
• Component/s: Ruby - Library

Environment

JRuby 1.6.8 using "--1.9" command line parameter.

• Patch Info: Patch Available
• Assignee: Unassigned
• Reporter: XB
• Created: 11/Oct/12 22:06
• Updated: 10/Jul/14 13:42
• Resolved: 10/Jul/14 13:42

Description

When setting a binary field of a Thrift object with some binary data (e.g. a string whose encoding is "ASCII-8BIT") and then serializing this object, the binary data is re-encoded. That is, it is encoded as if it were not a sequence of bytes but a sequence of characters, encoded using the ISO-8859-1 encoding. This assumed ISO-8859-1 sequence of characters is then converted into UTF-8 (by BinaryProtocol or CompactProtocol). This basically means that all bytes whose values are between 0x80 (inclusive) and 0x100 (exclusive) are converted into multi-byte sequences. This leads to data corruption.

Issue Links

• is superceded by: THRIFT-1766 [Ruby] Provide support for binary types