[THRIFT-5324] THttpClient sends previous message or partial message after http error

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.13.0
• Fix Version/s: 0.14.0
• Component/s: Go - Library
• Assignee: Unassigned
• Reporter: Ilya Morozov
• Created: 14/Dec/20 20:02
• Updated: 07/Mar/22 18:56
• Resolved: 07/Mar/22 18:56

Description

We noticed that there's an "out of order sequence response" error immediately after an http error or premature context closure. We are seeing a lot of these in production.

It seems like THttpClient does not reset it's internal buffer when http client returns an error, leaving the whole or partially read message in the buffer. On next call THttpClient sends the remains of previous message followed by current message, causing the server to execute it  and return the previous seqId hence the  "out of order sequence response" errors.

Here's a small test of THttpClient that demonstrates this issue: https://github.com/wheely/thrift/blob/c130f2452428540a4eefc4c422c8abc329ec837d/lib/go/thrift/http_client_test.go#L101-L147 .

First flush fails due to closed context, second flush succeeds but writes both messages to the server. In current thrift version this test fails with:

http_client_test.go:144: Received unexpected data: write 1write 2

 

Please let me know if you need a better example.

 

Issue Links

• links to: GitHub Pull Request #2293
• links to: GitHub Pull Request #2295