[THRIFT-1639] Java/Python: Serialization/Deserialization of double type using CompactProtocol

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8
• Fix Version/s: 0.9.2
• Component/s: Java - Library, Python - Library
• Assignee: Roger Meier
• Reporter: andrew watts
• Created: 28/Jun/12 20:50
• Updated: 19/Nov/13 17:06
• Resolved: 05/Nov/13 03:56

Description

Using the Compact Protocol double values are not properly serialized/deserialized with Java or Python.

I have a java server that writes a thrift blob to log file as base64 encoded string, then python reads the base64 encoded string from the log. During development I discovered double values are not deserialized properly.

In the mailing list there is speculation of mismatch between serialization/deserialization in the languages and recommended opening a ticket.

Example:

» java -cp .:../lib/libthrift-0.8.0.jar:../lib/slf4j-api-1.6.4.jar ThriftTest
fooObj.bar: 3.456
base64String: F9nO91PjpQtAAA==

» python ../python/tdouble_test.py
base64 string:  F9nO91PjpQtAAA==
foo_obj.bar:  -4.09406819342e+124    # expect 3.456

Thrift Definition:

struct FooObj {
    1: double bar
}

Java Code:

import org.apache.thrift.TException;
import org.apache.thrift.TSerializer;
import org.apache.thrift.protocol.TCompactProtocol;

import javax.xml.bind.DatatypeConverter;


public class ThriftTest {

   public static void main(String[] args) {

       final TSerializer serializer = new TSerializer(new
TCompactProtocol.Factory());

       // create a FooObj with double
       final FooObj fooObj = new FooObj(3.456);
       System.out.println("fooObj.bar: " + fooObj.bar);

       // serialize to bytes
       byte[] fooObjBlob = null;
       try {
           fooObjBlob = serializer.serialize(fooObj);
       } catch (TException e) {
           e.printStackTrace();
       }

       // encode to base64 string
       final String base64String =
DatatypeConverter.printBase64Binary(fooObjBlob);
       System.out.println("base64String: " + base64String);
   }

}

Python Code

#!/bin/env python

import base64

from thrift.protocol import TCompactProtocol
from thrift.TSerialization import deserialize

from foo.ttypes import FooObj


def main():

   protocol_factory = TCompactProtocol.TCompactProtocolFactory
   base64_string = 'F9nO91PjpQtAAA=='

   print 'base64 string: ', base64_string

   # deserialize the string back into an object
   foo_blob = base64.urlsafe_b64decode(base64_string)
   foo_obj = FooObj()
   deserialize(foo_obj, foo_blob, protocol_factory=protocol_factory())
   print 'foo_obj.bar: ', foo_obj.bar

if __name__ == '__main__':
   main()