Uploaded image for project: 'Thrift'
  1. Thrift
  2. THRIFT-1639

Java/Python: Serialization/Deserialization of double type using CompactProtocol

    XMLWordPrintableJSON

Details

    Description

      Using the Compact Protocol double values are not properly serialized/deserialized with Java or Python.

      I have a java server that writes a thrift blob to log file as base64 encoded string, then python reads the base64 encoded string from the log. During development I discovered double values are not deserialized properly.

      In the mailing list there is speculation of mismatch between serialization/deserialization in the languages and recommended opening a ticket.

      Example:

      » java -cp .:../lib/libthrift-0.8.0.jar:../lib/slf4j-api-1.6.4.jar ThriftTest
      fooObj.bar: 3.456
      base64String: F9nO91PjpQtAAA==
      
      » python ../python/tdouble_test.py
      base64 string:  F9nO91PjpQtAAA==
      foo_obj.bar:  -4.09406819342e+124    # expect 3.456
      

      Thrift Definition:

      struct FooObj {
          1: double bar
      }
      

      Java Code:

      import org.apache.thrift.TException;
      import org.apache.thrift.TSerializer;
      import org.apache.thrift.protocol.TCompactProtocol;
      
      import javax.xml.bind.DatatypeConverter;
      
      
      public class ThriftTest {
      
         public static void main(String[] args) {
      
             final TSerializer serializer = new TSerializer(new
      TCompactProtocol.Factory());
      
             // create a FooObj with double
             final FooObj fooObj = new FooObj(3.456);
             System.out.println("fooObj.bar: " + fooObj.bar);
      
             // serialize to bytes
             byte[] fooObjBlob = null;
             try {
                 fooObjBlob = serializer.serialize(fooObj);
             } catch (TException e) {
                 e.printStackTrace();
             }
      
             // encode to base64 string
             final String base64String =
      DatatypeConverter.printBase64Binary(fooObjBlob);
             System.out.println("base64String: " + base64String);
         }
      
      }
      

      Python Code

      #!/bin/env python
      
      import base64
      
      from thrift.protocol import TCompactProtocol
      from thrift.TSerialization import deserialize
      
      from foo.ttypes import FooObj
      
      
      def main():
      
         protocol_factory = TCompactProtocol.TCompactProtocolFactory
         base64_string = 'F9nO91PjpQtAAA=='
      
         print 'base64 string: ', base64_string
      
         # deserialize the string back into an object
         foo_blob = base64.urlsafe_b64decode(base64_string)
         foo_obj = FooObj()
         deserialize(foo_obj, foo_blob, protocol_factory=protocol_factory())
         print 'foo_obj.bar: ', foo_obj.bar
      
      if __name__ == '__main__':
         main()
      
      

      Attachments

        1. python_little_endian_double.diff
          0.6 kB
          Patrick Lawson

        Activity

          People

            roger Roger Meier
            andrewwatts andrew watts
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: