[THRIFT-4815] Golang thrift and Python don't write the same messages

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Not A Bug
• Affects Version/s: 0.11.0, 0.12.0
• Component/s: Go - Compiler, Go - Library, Python - Compiler, Python - Library
• Labels: newbie

Environment

Python:
Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)] on win32
Thrift version 0.11.0 tested.
Golang:
Thrift versions 0.12.0 and 0.11.0 tested
go version go1.11.4 windows/amd64
go env:
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\BUNNY\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\BUNNY\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\BUNNY\AppData\Local\Temp\go-build552047466=/tmp/go-build -gno-record-gcc-switches

• Assignee: James E. King III
• Reporter: Jack Flecher
• Created: 11/Mar/19 10:44
• Updated: 02/Mar/21 19:33
• Resolved: 11/Mar/19 12:43

Description

I'm trying to port a tool that relies on Thrift from Python to Golang, it is a client not a server.

The thrift protocol behaves inconsistent when comparing Python and Golang and it results in breaking applications.
What it boils down to is I have a struct called Message with a certain amount of fields and I have a sendMessage method that takes that message and sends it to a server which then processes that.
In Python I can simply make a Message object, set some user's userId as receiver and the text after which I can send the message using sendMessage()
In Golang I can do the exact same but I run into error responses from the server saying another field that isn't required cannot be found.

In Python this is the generated POST body sent to the server:
b'\x82!\x00\x0bsendMessage\x15\x00\x1c(!u218891b1a7af4d21ffe4918acbeb9a73\x88\x04test\x00\x00'
b'\x82!\x00\x0bsendMessage\x15\x00\x1c(!u218891b1a7af4d21ffe4918acbeb9a73\x88\x05test2\x00\x00'

In Golang the generated POST body sent to the server:
b'\x82!\x01\x0bsendMessage\x15\x00\x1c(\x04test\x88!u218891b1a7af4d21ffe4918acbeb9a73\x00\x00'
b'\x82!\x02\x0bsendMessage\x15\x00\x1c(\x05test2\x88!u218891b1a7af4d21ffe4918acbeb9a73\x00\x00'

As demonstrated above therre are two differences in the generated POST bodies but the one causing the error is the fact that the fields are in a different order than they are supposed to.

In Golang trying to reverse the order of the fields causes this POST body to be generated:
b'\x82!\x01\x0bsendMessage\x15\x00\x1c\xa8!u218891b1a7af4d21ffe4918acbeb9a73\x08\x04\x04test\x00\x00'
When commenting out all the fields I actually don't want to be written and then only reversing the functions that are actually writing data, the ones for the receiver and text fields the POST body looks like this:
b'\x82!\x01\x0bsendMessage\x15\x00\x1c(\x04test\x88!u218891b1a7af4d21ffe4918acbeb9a73\x00\x00'

To summarize, the Thrift protocol on Golang is broken in certain cases because fields are not written in the right order causing other implementatings of Thrift to stop serving Golang Thrift clients.

Issue Links

• is related to: THRIFT-4782 Provide a dynamic interop matrix driven by the cross test results